MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
c825e1d235
[Perf] Fuse the mamba align block-table gather into one Triton kernel, 7 kernels -> 1, 6x speedup on all shapes for this kernel ( #58737 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Signed-off-by: lifulu <lifulu@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: lifulu <lifulu@users.noreply.github.com >
2026-10-15 11:02:53 -05:00
Nicolò Lucchesi
30d4032363
[Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer ( #59211 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-05 16:44:14 +02:00
f7999d2e44
[Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates ( #59450 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-02 04:11:55 +00:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Jiangyun Zhu and OpenAI Codex
6632ed559e
[Perf][Attention] Avoid CPU-GPU sync in DCP sequence lengths ( #58169 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 16:28:43 +08:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
241e9391ed
[Bugfix][Attention] Stabilize sparse-MLA DCP for GLM PCP evals ( #55879 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Jason Yao <wsyjh8@gmail.com >
2026-09-15 09:58:30 -07:00
82a85dc1d2
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention ( #56305 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:29:41 +00:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00
Nick Hill
e6cb56337b
[Core] MRV2 support for fast-prefill ( #56145 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-10 10:36:04 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
Nick Hill
4aab2b0ebe
[Model Runner V2] Use MRV2 for all models by default ( #53183 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-27 11:59:34 -07:00
Misha Goin
f620499ee3
[Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata ( #53336 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-24 08:08:43 -07:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
Misha Goin
9ff7041b55
[Bugfix][Spec Decode] Use group geometry for FlashAttention metadata ( #53002 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-21 17:07:19 -07:00
yimdev and usberkeley
574e6a00ef
[Bugfix][Attention] Normalize FlashInfer prefill LSE before merging ( #52796 )
...
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com >
Co-authored-by: usberkeley <150880684+usberkeley@users.noreply.github.com >
2026-08-21 11:31:14 -04:00
Nick Hill
f97e502969
[Perf] Avoid more GPU<->CPU syncs on the model execution path ( #51738 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-11 19:30:34 -07:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
Michael Goin and OpenAI Codex
90329913e9
[UX] Reduce startup log noise ( #50590 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-31 13:04:34 -07:00
b6ff8a2f50
[Core] Add MRV2 virtual-batch PCP for MLA ( #46570 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-19 02:53:15 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f17be06fbe
[Perf] Optimize clamp to clamp_ ( #48143 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-16 18:41:07 -04:00
c5200d3565
[Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE ( #46076 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: GirasoleY <girasoleyang@gmail.com >
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-01 00:32:20 -04:00
a7732537f4
[Bugfix] Restore part of bugfix #42650 after accidental deletion in #43241 ( #47039 )
...
Signed-off-by: zhanda <zhandazhu@gmail.com >
Signed-off-by: Nikita Shapovalov <nikita@poolside.ai >
Co-authored-by: Zhanda Zhu <49645678+zhandaz@users.noreply.github.com >
Co-authored-by: Shang Wang <shangw@nvidia.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-06-30 11:07:59 -07:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
6deb05e0e4
[Core][Model] Gemma4: Unified FA4 for all layers + FlashAttention mm_prefix support ( #42175 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-06-09 17:45:39 -07:00
Micah Williamson
c9c1540e61
[ROCm][V2] Fix failed assertion in Llama models when using EAGLE with ROCM_AITER_FA ( #44936 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-06-09 13:30:52 -05:00
Giancarlo Delfin
ceb0111a90
[Model Runner V2][Spec Decode] Add Gemma4 MTP support ( #43241 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-06-04 00:51:06 +00:00
Zhanda Zhu and Shang Wang
c7624bea5e
[Bugfix] Source num_qo_heads from Attention layers in Flashinfer/Triton metadata builders ( #42650 )
...
Signed-off-by: zhanda <zhandazhu@gmail.com >
Co-authored-by: Shang Wang <shangw@nvidia.com >
2026-05-22 16:10:03 +00:00
roikoren755
737bfa3a43
[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash ( #41233 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-05-18 14:54:00 +03:00
Nick Hill
989c176c0a
[Perf][3/n] Eliminate GPU<->CPU syncs in attention impls ( #41434 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-07 19:44:24 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Nick Hill
fe85a92e86
[Core] Avoid seq_lens_cpu GPU->CPU sync ( #40654 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-24 00:35:55 +00:00
bcc6f67447
[Bugfix] Use null block (0) for padded block table entries ( #35431 )
...
Signed-off-by: SandishKumarHN <sandish@fb.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-30 14:02:51 -07:00
Or Ozeri
7cc302dd87
[kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models ( #37853 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-03-27 08:38:33 +03:00
Chuan (Richard) Li
cb2263218e
[Bugfix][Minor] Fix potential NameError in mamba backend selector and misc typos ( #35886 )
...
Signed-off-by: Li <chuali@amd.com >
2026-03-26 11:59:24 -04:00
Lucas Wilkinson
e1d85e5c24
[Attention] Support distinguishing between short extends and decodes ( #37303 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-03-20 10:49:36 -07:00
Woosuk Kwon
006aea17d7
[BugFix] Remove incorrect assert in split_decodes_and_prefills ( #36553 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-03-09 20:02:02 -07:00
Huamin Li and Cyrus Leung
1fe462168c
[perf] Avoid dtype promotion sync in mamba_get_block_table_tensor ( #34870 )
...
Signed-off-by: Huamin Li <3ericli@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-02-20 06:21:56 -08:00
Woosuk Kwon
0916e7960b
[GDN] Use CPU tensors to build GDN metadata ( #34498 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-02-13 01:24:45 -08:00
Benjamin Chislett
af3162d3aa
[Spec Decode] Unified Parallel Drafting ( #32887 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
2026-02-05 12:37:18 -05:00
Harry Huang and Chen Zhang
5206e5e28c
[V1][Hybrid] Mamba Prefix Caching with align mode ( #30877 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Chen Zhang <zhangch99@outlook.com >
Co-authored-by: Chen Zhang <zhangch99@outlook.com >
2026-01-23 09:56:48 -08:00
tianshu-Michael-yu
13d8746c54
[Feature]: Remove DtoH Copy for lfm2_vl On Default Stream ( #32815 )
...
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com >
2026-01-23 13:20:30 +00:00
Tomas Ruiz
4a5299c93f
feat: spec decode with draft models ( #24322 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-01-19 16:05:46 -05:00
Matthew Bonanni
98f60e5acb
[6/N][Attention] Move utils to more appropriate locations ( #32215 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-13 05:38:52 -08:00
Matthew Bonanni
20228cb851
[3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py ( #32054 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-12 09:13:56 -08:00
Asaf Joseph Gardin
8fb2c135be
[Bugfix] Fix stale SSM state for new Mamba requests scheduled as decode ( #32118 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-01-12 17:02:38 +00:00
Lucas Wilkinson
da6709c9fe
[Misc] Delay deprecation of CommonAttentionMetadata properties ( #32074 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-01-09 21:06:44 -08:00