56 Commits
Author SHA1 Message Date
MOVZXandHinata 61fbe0b865 [Mini-VLLM] Remove unreferenced definitions found by static scan
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.

- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
  reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
  unused BailingLinearAttentionBackend, and the sparse flash-attn entry
  points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
  cpu_mla_extend, cpu_mla_get_scheduler_metadata,
  gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
  gptq_quantize_weights, get_and_maybe_dequant_weights,
  marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
  ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
  input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
  run_dp_sharded_vision_model, fast_topk, get_layer_index,
  is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
  selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
  split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
  is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
  create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
  vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
  neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.

Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-19 21:39:56 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-18 19:19:56 +07:00
026f6ab268 [Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857)
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-17 09:37:56 +00:00
Harry MellorClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kevin H. Luu
b965ee564d [Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 01:08:08 -07:00
Yongye ZhuandClaude Opus 5.5 a6009ea85b [Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 23:24:43 -07:00
Ilya Markov a281ad32f0 [EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
2026-10-12 10:03:09 -04:00
Warren Deng 59ffd73e85 [Bugfix][Kernel] Restore FP fusion for indexer Q quantization (#60716)
Signed-off-by: Warren Deng <warrdeng@meta.com>
2026-10-09 09:46:41 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
Yida WengandGrok 17e9295dd5 [BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size (#59271) (#59373)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Co-authored-by: Grok <grok@x.ai>
2026-09-30 15:09:11 +00:00
Kevin H. Luu 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) 2026-09-26 11:37:36 -04:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 23:26:50 -07:00
44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-23 22:45:23 -07:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-20 21:51:13 +08:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:29:11 +01:00
Julien DebacheandHarry Mellor 2bdae2a5a8 [fix] Mistral-Large-3 accuracy regression on main (#57563)
Signed-off-by: jdebache <jdebache@nvidia.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-09-18 14:19:16 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Jared WenandClaude Fable 5.1 1257512305 [DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254)
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 14:07:14 -07:00
linitra24 dd4c841070 [LoRA] Add LoRA support for DeepSeek-V4 Flash Vision (#55897)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
2026-09-13 07:10:54 -07:00
Roberto L. CastroandCodex 120ec4ebd2 [5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-12 00:05:59 -04:00
Andreas KaratzasandOpenAI Codex 9dd969da09 [Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-11 09:17:30 -07:00
9b959b8657 [Model] DeepSeek-V4.1-Flash Model Definitions (#56228)
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-10 20:16:26 +08:00
Li, Jiang c3ccc0e957 [Model] Add DeepSeek-V4 CPU backend (#55355)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-09-10 14:14:48 +08:00
jimmy-adams 62f3bf58a5 [Bugfix][ROCm][DSv4] Skip launch_pdl=True JIT warmup when PDL is unsupported (#56035)
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
2026-09-09 13:50:41 +00:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-08 10:25:48 -04:00
aoshen02andaoshen02 7fbd44cbe0 [Bugfix][DSv4] Seed the -1 sentinel in the prefill sparse index workspace (#55299)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
2026-09-05 18:31:04 +08:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-02 11:09:09 -07:00
Isotr0py 1d8d7a3965 [Bugfix] Fix RoPE construction for deepseek-v4 sparse SWA layers (#54815)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-01 20:48:50 -07:00
Andreas Karatzas 6b68db441e [ROCm] Fix DeepSeek V4 indexer numerics and coverage (#50803)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-08-20 09:34:00 +00:00
Woosuk KwonandOpenAI Codex f1178f3a06 Revert DSv4 eager workspace reuse (#52836)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-18 17:02:59 -07:00
ef43e3101b [ROCm][DSV4][Perf] Optimize Triton sparse-MLA decode on gfx950 (#52212)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cursor Grok 4.6 <cursoragent@cursor.com>
2026-08-16 12:16:46 -07:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 83f591d7f6 [Perf][DSV4] Optimize global top-k index kernel with compile-time constants (#51967)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-16 23:24:14 +08:00
Chauncey 836aac92ff [Perf][DSV4] Optimize sparse top-k metadata kernels for higher prefill throughput (#52084)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-08-16 21:15:47 +08:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
Wentao Ye 837eae6458 [DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-30 11:39:07 -04:00
+13 aeeb36b1f1 [New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
2026-07-30 03:49:58 -07:00
Avery MiaoandKunshang Ji c67650f04b [XPU][DeepSeekV4]Add DeepSeek-V4 fuse_index_q SYCL kernel path (#45991)
Signed-off-by: Avery Miao <avery.miao@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-07-21 21:22:34 +08:00
Roberto L. Castroandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 7ca017778f [Feat][Perf] Add new warmup infrastructure for JITs (#47451)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 13:21:55 -07:00
Xiaochang WuKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
823eaf667d [XPU] FP8 o_proj with fp8_bmm and load-time scale transpose (#48334)
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 16:32:03 +08:00
kliuae eb33ff34dd [ROCm][Perf] DSv4 two-stage compressor kernel for HCA prefill (#47718)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-07-15 23:31:51 +00:00
Jeff (Junze) MaandLucas Wilkinson 80eb01e93d [Bugfix] DSV4 TP16 garbage output (#47493)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-07-07 21:04:33 -07:00
akii96 aa8bb5562e [ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
2026-07-01 17:33:55 +08:00
44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-06-22 11:54:14 -07:00
Jee Jee Li 22cc891108 [Kernel] Add PDL support for DeepGEMM kernel (#46006)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-06-18 20:49:01 +08:00
afdcbd5d39 [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
2026-06-18 12:21:14 +00:00
Micah Williamson e945169207 Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999) 2026-06-17 22:59:48 -07:00
Jee Jee Li 4403af8fb5 [Kernel] Add PDL support for DeepGEMM kernel (#42996)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-06-17 20:37:17 -07:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00
Woosuk Kwon 517e74a964 [DSV4] Refactor RoPE initialization (#44262)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-02 01:26:58 +00:00