meta: title: "DeepSeek-V4-Flash-Vision-Exp" slug: "deepseek-v4-flash-vision-exp" provider: "DeepSeek" description: "DeepSeek's first experimental multimodal V4 model — the V4-Flash MoE backbone plus a 32-layer vision tower, 1M context, and a fused DSpark draft module." date_added: 2026-09-01 date_updated: 2026-09-14 tasks: - text - multimodal performance_headline: "First multimodal V4 — 285B/13B MoE scoring 83.5% on OCRBench from a single GB200 NVL4 tray" related_recipes: - "deepseek-ai/DeepSeek-V4-Flash" - "deepseek-ai/DeepSeek-V4-Pro" # The vision path is unreleased — it lives in the open support PR # vllm-project/vllm#54566 (see model.install below). GB200 is the only # NVIDIA hardware with a published end-to-end run on that branch: TP4 + EP # on one NVL4 tray, FP8 KV cache, block size 256, DSpark k=3, full # 1000-sample OCRBench at 83.5% with zero request errors. # Atlas A5 8-card mixed serving of the native FP4+FP8 Vision-Exp checkpoint # (DP4+EP, TP=1, no speculative decoding). 950PR verified 2026-09-10 on # vLLM Ascend; 950DT uses the same command (4 of 8 NPUs). See the hardware # note for the required hash-router bias fix. # AMD MI350X (gfx950): ROCm vision merged in vllm-project/vllm#55107 (2026-09-11) # on the #54566 base. Verified text+vision on stock ROCm nightly at single-node # TP2 and TP4, with DSpark k=3 (needs VLLM_USE_V2_MODEL_RUNNER=1). See # "Launching on AMD" in the guide. default_hardware: gb200 hardware: gb200: verified mi350x: verified ascend_950dt_8x: verified ascend_950pr_8x: verified model: model_id: "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp" # Support targets the next release; nothing has shipped yet, so the pinned # image below is the only runnable path. min_vllm_version: "0.30.0" nightly_required: true docker_image: nvidia: "vllm/vllm-openai:deepseekv4-flash-vision" amd: "vllm/vllm-openai-rocm:nightly" install: # Docker first so it is the default tab. The official wheel can't run this # checkpoint: released vLLM prior to v0.30.0 routes it to the text-only # DeepseekV4ForCausalLM, which then fails on the vision/aligner tensors. # v0.30.0 already resolves it to DeepseekV4ForConditionalGeneration (verified on the ROCm image, 2026-10-02). docker: note: "NVIDIA: use the dedicated DeepSeek-V4-Flash-Vision image until support lands in the standard vLLM image. AMD: vllm/vllm-openai-rocm:v0.30.0 (ROCm 7.2.3) or a nightly >= 2026-09-11 (includes #55107); on v0.30.0 also pass --compilation-config '{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}' (see the AMD section). Ascend 950PR/950DT: pin quay.io/ascend/vllm-ascend:v0.23.0-a5 and overlay vllm-project/vllm-ascend#16259." pip: false architecture: moe parameter_count: "285B" active_parameters: "13B" context_length: 1048576 flashinfer_autotune: true base_args: - "--kv-cache-dtype" - "fp8" - "--block-size" - "256" features: tool_calling: description: "DeepSeek V4 tool-call parsing with automatic tool choice." args: - "--tool-call-parser" - "deepseek_v4" - "--enable-auto-tool-choice" reasoning: description: "Enable reasoning/thinking mode with the DeepSeek V4 reasoning parser." args: - "--reasoning-parser" - "deepseek_v4" - "--reasoning-config" - '{"reasoning_parser":"deepseek_v4","reasoning_start_str":"","reasoning_end_str":""}' # Empty start/end strings make the Ascend chat path emit content=null # for a full completion. Keep the parser, drop the override. hardware_overrides: npu: args: - "--reasoning-parser" - "deepseek_v4" # Single method, so the boolean form (not `modes`): the checkpoint's fused # DSpark module is the only drafter with a published run on this branch. Left # OUT of opt_in_features on purpose — the verified GB200 configuration had # DSpark on, and DeepSeek's own model card enables it too, so the default # builder state reproduces the tested command. The checkpoint also carries # 3 MTP layers (`num_nextn_predict_layers: 3`); that path is untested here. spec_decoding: description: "DSpark self-drafting from the checkpoint's fused draft module — 3 draft tokens, probabilistic sampling, adaptive verification. Mean acceptance length 2.99 tokens/forward on mixed text+image traffic." args: - "--speculative-config" - '{"method":"dspark","model":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","num_speculative_tokens":3,"draft_sample_method":"probabilistic","enable_adaptive_verification":true}' # 950 has no working MTP or DSpark path yet. Empty args so a toggle-on # still emits no --speculative-config. Default-off via # hardware_opt_in_features below — GB200 keeps DSpark on. hardware_overrides: # ROCm: the DeepseekV4 indexer backend rejects adaptive verification, so it # is forced false (DSpark still drafts k=3). VLLM_USE_V2_MODEL_RUNNER=1 # (hardware_overrides.amd.extra_env) is also required. amd: args: - "--speculative-config" - '{"method":"dspark","model":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","num_speculative_tokens":3,"draft_sample_method":"probabilistic","enable_adaptive_verification":false}' npu: args: [] # Spec decoding stays default-on everywhere except the A5 8-card pills. hardware_opt_in_features: ascend_950pr_8x: - spec_decoding ascend_950dt_8x: - spec_decoding variants: default: precision: fp8 # Mixed FP4 experts + FP8 remainder, so the bytes-per-param table # underestimates: sized from the real checkpoint (167.8 GB x 1.2). vram_minimum_gb: 202 description: "Native FP4+FP8 mixed checkpoint (MoE experts FP4, attention/norm/router FP8) with a BF16 vision tower, aligner, and the fused DSpark draft module." hardware_overrides: # Exact-GPU pin: computeDockerMeta has no Huawei brand, so the model-level # docker_image string only affects NVIDIA. Hide pip — official wheels # have no NPU kernels. ascend_950pr_8x: docker_image: "quay.io/ascend/vllm-ascend:v0.23.0-a5" install: pip: false ascend_950dt_8x: docker_image: "quay.io/ascend/vllm-ascend:v0.23.0-a5" install: pip: false compatible_strategies: - single_node_tp - single_node_tep - single_node_dep - multi_node_tep - multi_node_dep # TP + EP is the verified layout (TP4 + EP on one GB200 NVL4 tray). default_strategy: single_node_tep # Atlas A5 mixed serving is DP+EP only. The native checkpoint has 64 # attention heads; 950's FP8 DSA operator accepts local num_heads_q of # 64 or 128, so TP must stay 1. TEP/TP and intra-node PD are not offered. strategy_hardware: single_node_tp: npu: unsupported single_node_tep: npu: unsupported multi_node_tep: npu: unsupported multi_node_dep: npu: unsupported # Shared by 950DT 8x (8×96 GB) and 950PR 8x (8×128 GB). Mixed serving uses # 4 of the 8 NPUs (DP4+EP, TP=1). HCCL_BUFFSIZE=1024 is required for MC2 # dispatch. last-wins shadows the DEP builder's --data-parallel-size 8 # and the recipe-level --block-size 256 / --kv-cache-dtype fp8. hardware_overrides: amd: extra_args: - "--tensor-parallel-size" - "2" - "--gpu-memory-utilization" - "0.9" - "--max-model-len" - "32768" extra_env: VLLM_ROCM_USE_AITER: "1" VLLM_USE_BREAKABLE_CUDAGRAPH: "1" VLLM_USE_V2_MODEL_RUNNER: "1" npu: extra_args: - "--data-parallel-size" - "4" - "--block-size" - "128" - "--kv-cache-dtype" - "auto" - "--tokenizer-mode" - "deepseek_v4" - "--safetensors-load-strategy" - "prefetch" - "--async-scheduling" - "--max-model-len" - "8192" - "--max-num-seqs" - "16" - "--max-num-batched-tokens" - "2048" - "--gpu-memory-utilization" - "0.9" - "--api-server-count" - "1" - "--compilation-config" - '{"cudagraph_mode":"FULL_DECODE_ONLY"}' - "--additional-config" - '{"enable_cpu_binding": true, "multistream_overlap_shared_expert": true, "enable_shared_expert_dp": true}' extra_env: PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True" OMP_PROC_BIND: "false" OMP_NUM_THREADS: "10" HCCL_BUFFSIZE: "1024" HCCL_OP_EXPANSION_MODE: "AIV" TASK_QUEUE_ENABLE: "1" ASCEND_RT_VISIBLE_DEVICES: "0,1,2,3" hardware_notes: ascend_950dt_8x: title: "vLLM Ascend A5 image required" body: "Mixed serving on one 8-card 950DT node uses 4 of 8 NPUs (DP4+EP, TP=1). Same command as 950PR. Pin quay.io/ascend/vllm-ascend:v0.23.0-a5 — the NVIDIA pip wheel cannot run here. Set HCCL_BUFFSIZE=1024 before launch. Native Flash-Vision checkpoints need the hash-router bias skip in vllm-project/vllm-ascend#16259 — without it load_weights aborts on model.layers.0.mlp.gate.e_score_correction_bias." ascend_950pr_8x: title: "vLLM Ascend A5 image required" body: "Mixed serving on one 8-card 950PR node uses 4 of 8 NPUs (DP4+EP, TP=1). Same command as 950DT. Pin quay.io/ascend/vllm-ascend:v0.23.0-a5 and set HCCL_BUFFSIZE=1024 before launch. Native Flash-Vision checkpoints need the hash-router bias skip in vllm-project/vllm-ascend#16259 — without it load_weights aborts on model.layers.0.mlp.gate.e_score_correction_bias." guide: | ## Overview DeepSeek-V4-Flash-Vision-Exp is DeepSeek's first multimodal model in the V4 family. It keeps the DeepSeek-V4-Flash language backbone — 43 layers, 256 routed experts with 6 active per token, Compressed Sparse Attention plus manifold-constrained hyper-connections, a 1,048,576-token context window — and adds a 32-layer / 1024-dim ViT with a two-layer aligner (~0.5B parameters on top of the 284B backbone). Weights are **FP4+FP8 mixed**: MoE experts in FP4, the remaining attention / norm / router params in FP8, vision tower in BF16. The checkpoint is 48 shards / ~168 GB and carries the fused **DSpark** draft module. DeepSeek reports it beats DeepSeek-V4-Flash-0731 on multimodal agent benchmarks (ApexBench 36.5 vs 26.2, Agents' Last Exam 27.3 vs 25.2) while holding text-agent parity (Terminal Bench 2.1 at 83.9, DeepSWE at 59.3). ## Prerequisites Vision support is not in a stable vLLM release yet — it lives in [vllm-project/vllm#54566](https://github.com/vllm-project/vllm/pull/54566). Until that lands, use the pinned `vllm/vllm-openai:deepseekv4-flash-vision` image; the official wheel routes this checkpoint to the text-only class and fails on the vision tensors. - **Weights:** ~168 GB on disk (48 shards). Budget ~202 GB of VRAM before KV cache. - **Hardware:** NVIDIA (verified on GB200) and AMD MI350X (gfx950); XPU still has no vision implementation. - **Tokenizer / chat template:** nothing to pass. The repo ships no Jinja template (prompt encoding lives in `encoding/`); vLLM resolves `--tokenizer-mode` to `deepseek_v4` automatically, which is what turns OpenAI content blocks into `<｜deepseek_image｜>` placeholders. ## Launching the server ### Verified configuration — one GB200 NVL4 tray (TP4 + EP) ```bash export VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS="trtllm_fp4_block_scale_moe,flashinfer::trtllm_fp4_block_scale_moe" vllm serve deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \ --tensor-parallel-size 4 \ --enable-expert-parallel \ --kv-cache-dtype fp8 \ --block-size 256 \ --max-model-len 32768 \ --speculative-config '{"method":"dspark","model":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","num_speculative_tokens":3,"draft_sample_method":"probabilistic","enable_adaptive_verification":true}' \ --allowed-local-media-path /mnt/lustre ``` Three things there are environment-specific, which is why the builder above leaves them out: - `VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS` excludes the FP4 block-scale MoE op from FlashInfer's startup autotuning. Export it if autotuning that op fails or costs more startup time than it earns back on your FlashInfer build; it is a tuning knob, not a requirement. The Advanced row's `--no-enable-flashinfer-autotune` turns autotuning off wholesale instead. - `--max-model-len 32768` bounds KV cache for the benchmark. The checkpoint advertises 1M tokens; that is **not** what was measured, and 1M-token KV needs far more memory than the weights do. Raise it deliberately, or use the Advanced row's `--max-model-len auto`. - `--allowed-local-media-path` is only needed for `file://` image URLs. Drop it if clients send `http(s)://` or base64 data URLs. ### 8-GPU nodes (H200 / B200 / B300) Same **Tensor + Expert Parallel** strategy; the builder emits `--tensor-parallel-size 8 --enable-expert-parallel`. The language backbone is the one the [DeepSeek-V4-Flash](/deepseek-ai/DeepSeek-V4-Flash) recipe already runs there, but the vision path has no published run on those GPUs — expect to tune `--gpu-memory-utilization` and `--max-num-seqs`. The Blackwell `--attention_config.indexer_kv_dtype mxfp4` / `--moe-backend deep_gemm_mega_moe` tweaks the text recipe applies are deliberately not set here: the verified vision run did not use them. ### AMD MI350X (gfx950) ROCm vision support is merged (#55107 on the #54566 base); a `vllm/vllm-openai-rocm:nightly` built on/after 2026-09-11 runs the checkpoint natively. Three env vars are required: - `VLLM_ROCM_USE_AITER=1` — the sparse-attention indexer's ROCm path runs only on AITER; without it the engine aborts at KV-cache profiling. - `VLLM_USE_BREAKABLE_CUDAGRAPH=1` — the multimodal wrapper is not torch-compiled, so the default `FULL_AND_PIECEWISE` capture cannot attach without breakable graphs. - `VLLM_USE_V2_MODEL_RUNNER=1` — required for DSpark (the V1 ROCm runner refuses it); harmless for plain serving. Verified single-node launch: ```bash docker run -d --name dsv4-vision \ --ipc host --network host --privileged --shm-size 128g \ --device /dev/kfd --device /dev/dri --group-add video \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -e VLLM_ROCM_USE_AITER=1 \ -e VLLM_USE_BREAKABLE_CUDAGRAPH=1 \ -e VLLM_USE_V2_MODEL_RUNNER=1 \ vllm/vllm-openai-rocm:nightly \ deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \ --tensor-parallel-size 2 \ --kv-cache-dtype fp8 --block-size 256 \ --gpu-memory-utilization 0.9 \ --max-model-len 32768 ``` Observed backends: AITER Flash-Attention for the ViT, DeepSeek `fp8_ds_mla` KV cache, FP8 Lightning-Indexer cache, `DEEPSEEK_SPARSE_SWA` (block 256), and the `ROCM_FLASHMLA_SPARSE_DSV4` MLA backend. TP2 loads 75.2 GiB/card (1.76M-token KV pool at 131K context); TP4 loads 38.8 GiB/card. **Stable v0.30.0 and single-card TP1** (verified 2026-10-02 on one MI350X). The stable `vllm/vllm-openai-rocm:v0.30.0` image (ROCm 7.2.3) already contains #54566 and #55107 and runs the checkpoint natively — text, vision and DSpark — with the same three env vars. On v0.30.0 only, also pass `--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}'`: v0.30.0's ROCm platform still defaults DeepSeek V4 on gfx950 + ModelRunner V2 to eager (the #52644 workaround, removed by #57229 after v0.30.0 was branched), so without it decode runs without CUDA graphs. Batch 1 at TP1: 12 tok/s eager vs 94 tok/s with `FULL_DECODE_ONLY`, and 137–164 tok/s with DSpark (k=3, adaptive verification off per the AMD override). Against eager in the same run, a 25-item known-answer check scored 25/25 and decode-vs-prefill logprobs agreed within eager's range. Nightlies after 2026-09-17 and v0.31+ don't need the override. TP1 fits one 288 GB card (147.9 GiB of weights). ## Reasoning modes Same three-tier control as the rest of the V4 family, driven from `chat_template_kwargs` rather than a flag — `encoding/encoding_dsv4.py` names the effort levels `low` / `high` / `max`. DeepSeek evaluated this model at `reasoning_effort: "max"` with `temperature = 1.0, top_p = 0.95`. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") model = "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp" resp = client.chat.completions.create( model=model, messages=[{"role": "user", "content": "What is 17*19? Return only the final integer."}], temperature=1.0, top_p=0.95, extra_body={ "chat_template_kwargs": {"thinking": True, "reasoning_effort": "max"}, }, ) print(resp.choices[0].message.reasoning) print(resp.choices[0].message.content) ``` ## Client usage — image input Standard OpenAI multi-part content. Multiple images per request are allowed with no cap, so the practical limit is `--max-model-len`. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") resp = client.chat.completions.create( model="deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", messages=[{ "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}}, {"type": "text", "text": "Read the y-axis label and the highest bar's value."}, ], }], max_tokens=512, ) print(resp.choices[0].message.content) ``` ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "messages": [{ "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}}, {"type": "text", "text": "Describe the image."} ] }], "max_tokens": 512 }' ``` Image preprocessing is fixed by the checkpoint config (the processor takes no kwargs): each image costs at most 387 prompt tokens (`vision_max_n_token: 384` plus the compressor-alignment pad), images below `vision_min_pixels: 147456` are upscaled, and aspect ratios beyond 8:1 are cropped. An image's exact token count depends on where it sits in the prompt, since the compressor pads each image to a 4-token block boundary. ## Speculative decoding DSpark drafts from the module fused into the checkpoint, so `--speculative-config` names the target repo as its own `model` — there is no separate draft repo to download. Measured over the full OCRBench run on GB200 (mixed text+image traffic): **2.99** tokens/forward mean acceptance length, **66.3%** overall acceptance (83.9% / 66.5% / 50.7% by draft position). Image tokens in the prompt do not degrade acceptance — the drafter reads image content through the target's hidden states. Stay at `num_speculative_tokens: 3` unless you verify acceptance yourself; the checkpoint's trained DSpark block width is 5, but 3 is the depth with published numbers. ## Benchmarking OCRBench, full 1000 samples, on the verified GB200 configuration: **835/1000 (83.5%)**, 0 request errors, 312.6 s (~3.2 samples/s). Strongest categories are Key Information Extraction (92.5%), Doc-oriented VQA (90.0%) and Scene Text VQA (89.5%); weakest are handwritten math (49.0%) and handwriting recognition (58.0%). For throughput: ```bash vllm bench serve \ --model deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \ --dataset-name random \ --random-input-len 4096 \ --random-output-len 1024 \ --num-prompts 200 ``` ## Limitations - **Pre-release.** Flags and defaults can change before #54566 lands; re-check before deploying, and keep the image pinned. - **XPU has no vision path** — it raises `NotImplementedError` at model construction. - **GB200 (NVIDIA) and MI350X (AMD) are the hardware with a verified vision run.** Every other pill is the repo's fail-open default, not a tested claim. - **PD disaggregation is not offered**, and the CPU/filesystem KV-offload pills stay off — neither has a verified run to record. The Mooncake KV-store pills follow the repo's fail-open default and are offered, but untested here. - **1M context is advertised, not measured.** The reference run used 32K. ## References - [Model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) - [vLLM support PR #54566](https://github.com/vllm-project/vllm/pull/54566) - [vLLM ROCm vision PR #55107](https://github.com/vllm-project/vllm/pull/55107) - [DeepSeek-V4-Flash recipe](/deepseek-ai/DeepSeek-V4-Flash) — the text-only sibling - [DeepSpec (DSpark)](https://github.com/deepseek-ai/DeepSpec)