MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
MOVZX and Hinata
e61dcb9143
[Mini-vLLM] SM86-only cut: models, backends, Rust, tests
...
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-18 19:19:56 +07:00
026f6ab268
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint ( #58857 )
...
Signed-off-by: git bisector <gitbisector@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-17 09:37:56 +00:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Yida Weng and Grok
17e9295dd5
[BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size ( #59271 ) ( #59373 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Co-authored-by: Grok <grok@x.ai >
2026-09-30 15:09:11 +00:00
Kevin H. Luu
7d8c5fe9a9
[Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata ( #58499 )
2026-09-26 11:37:36 -04:00
Yongye Zhu and Claude Opus 5.5
5ff3bbfb08
[Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA ( #58621 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 23:26:50 -07:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00
Isotr0py
27757dde02
[DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash ( #56625 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-20 21:51:13 +08:00
Harry Mellor and Claude Opus 5
8a5cf54387
[Multimodal] Type dummy options per modality ( #57576 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 17:29:11 +01:00
Julien Debache and Harry Mellor
2bdae2a5a8
[fix] Mistral-Large-3 accuracy regression on main ( #57563 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-18 14:19:16 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Jared Wen and Claude Fable 5.1
1257512305
[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer ( #56254 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-15 14:07:14 -07:00
linitra24
dd4c841070
[LoRA] Add LoRA support for DeepSeek-V4 Flash Vision ( #55897 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-09-13 07:10:54 -07:00
Roberto L. Castro and Codex
120ec4ebd2
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels ( #53566 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 00:05:59 -04:00
Andreas Karatzas and OpenAI Codex
9dd969da09
[Model][ROCm] Enable DeepSeek V4 Vision ( #55107 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-11 09:17:30 -07:00
9b959b8657
[Model] DeepSeek-V4.1-Flash Model Definitions ( #56228 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-10 20:16:26 +08:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
jimmy-adams
62f3bf58a5
[Bugfix][ROCm][DSv4] Skip launch_pdl=True JIT warmup when PDL is unsupported ( #56035 )
...
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-09-09 13:50:41 +00:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
aoshen02 and aoshen02
7fbd44cbe0
[Bugfix][DSv4] Seed the -1 sentinel in the prefill sparse index workspace ( #55299 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-09-05 18:31:04 +08:00
Isotr0py
1356635d83
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support ( #54566 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-02 11:09:09 -07:00
Isotr0py
1d8d7a3965
[Bugfix] Fix RoPE construction for deepseek-v4 sparse SWA layers ( #54815 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-01 20:48:50 -07:00
Andreas Karatzas
6b68db441e
[ROCm] Fix DeepSeek V4 indexer numerics and coverage ( #50803 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-20 09:34:00 +00:00
Woosuk Kwon and OpenAI Codex
f1178f3a06
Revert DSv4 eager workspace reuse ( #52836 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 17:02:59 -07:00
ef43e3101b
[ROCm][DSV4][Perf] Optimize Triton sparse-MLA decode on gfx950 ( #52212 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Cursor Grok 4.6 <cursoragent@cursor.com >
2026-08-16 12:16:46 -07:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
83f591d7f6
[Perf][DSV4] Optimize global top-k index kernel with compile-time constants ( #51967 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-16 23:24:14 +08:00
Chauncey
836aac92ff
[Perf][DSV4] Optimize sparse top-k metadata kernels for higher prefill throughput ( #52084 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-16 21:15:47 +08:00
Gabriel Wu
97388c44f9
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark ( #51538 )
2026-08-15 07:01:32 -07:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
df71917cf1
[DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. ( #49236 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
Wentao Ye
837eae6458
[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement ( #50298 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:39:07 -04:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Avery Miao and Kunshang Ji
c67650f04b
[XPU][DeepSeekV4]Add DeepSeek-V4 fuse_index_q SYCL kernel path ( #45991 )
...
Signed-off-by: Avery Miao <avery.miao@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-21 21:22:34 +08:00
Roberto L. Castro and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7ca017778f
[Feat][Perf] Add new warmup infrastructure for JITs ( #47451 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 13:21:55 -07:00
823eaf667d
[XPU] FP8 o_proj with fp8_bmm and load-time scale transpose ( #48334 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 16:32:03 +08:00
kliuae
eb33ff34dd
[ROCm][Perf] DSv4 two-stage compressor kernel for HCA prefill ( #47718 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-07-15 23:31:51 +00:00
Jeff (Junze) Ma and Lucas Wilkinson
80eb01e93d
[Bugfix] DSV4 TP16 garbage output ( #47493 )
...
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-07-07 21:04:33 -07:00
akii96
aa8bb5562e
[ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 ( #46730 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-07-01 17:33:55 +08:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
Jee Jee Li
22cc891108
[Kernel] Add PDL support for DeepGEMM kernel ( #46006 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-18 20:49:01 +08:00
afdcbd5d39
[ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X ( #45681 )
...
Signed-off-by: ganyi <ygan@amd.com >
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: ganyi <ygan@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Jin Tao <jintao12@amd.com >
2026-06-18 12:21:14 +00:00
Micah Williamson
e945169207
Revert "[Kernel] Add PDL support for DeepGEMM kernel" ( #45999 )
2026-06-17 22:59:48 -07:00
Jee Jee Li
4403af8fb5
[Kernel] Add PDL support for DeepGEMM kernel ( #42996 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-17 20:37:17 -07:00
Yongye Zhu
b5235fca2e
[DSv4] Adding TRTLLM gen attention kernel ( #43827 )
2026-06-04 07:35:09 -07:00
Woosuk Kwon
517e74a964
[DSV4] Refactor RoPE initialization ( #44262 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-06-02 01:26:58 +00:00