MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
Luca Motz and Claude Opus 5.5
2d09fdcdc2
[Quantization] Auto-disable DeepGEMM for Kolibri 1 on SM12x ( #61274 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 12:59:50 +00:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Tony Lin
c42f5285ec
[Bugfix] prioritize architecture capability before DeepGEMM availability check ( #58073 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
2026-09-22 11:38:04 -04:00
41f9104fa6
[DSv4.1] Integrate Mega-Gate from DeepGEMM ( #56266 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-17 12:06:36 -07:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Harry Mellor and Claude Opus 5
5f203baedb
[Docs] Split slash-combined docstring parameters ( #57141 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 02:16:32 -07:00
Jared Wen and Claude Fable 5.1
1257512305
[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer ( #56254 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-15 14:07:14 -07:00
bb55077411
[Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255 ) ( #56962 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-15 04:01:06 -07:00
Yongye Zhu and Claude Opus 5
8263ea12bd
[Build] Move DeepGEMM pin to the vLLM fork and update the Mega MoE call convention ( #56876 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-15 00:57:18 -07:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
Zhuobin Huang and nodeeeeee
63a9a5010a
[Attention][DSA] Take the native decode path for MTP=3 on SM90 ( #52164 )
...
Signed-off-by: zobinHuang <zobin1999@gmail.com >
Co-authored-by: nodeeeeee <zhangkai.nodeee@gmail.com >
2026-08-14 10:09:51 +00:00
7f7a32cfec
[Spec Decode] DSpark confidence-scheduled verification ( #47808 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-12 12:27:48 -07:00
19d61b1273
[Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 ( #51359 )
...
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com >
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 14:27:50 -04:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
Jee Jee Li
22cc891108
[Kernel] Add PDL support for DeepGEMM kernel ( #46006 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-18 20:49:01 +08:00
Micah Williamson
e945169207
Revert "[Kernel] Add PDL support for DeepGEMM kernel" ( #45999 )
2026-06-17 22:59:48 -07:00
Jee Jee Li
4403af8fb5
[Kernel] Add PDL support for DeepGEMM kernel ( #42996 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-17 20:37:17 -07:00
Wentao Ye
9c7c74bf10
[Log] Update deepgemm log ( #45857 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-17 15:34:22 +00:00
Wei Zhao and Wei Zhao
2427094152
[Feature] Support EPLB for DeepSeek v4 Mega Moe ( #43339 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-lyris01.lyris.clusters.nvidia.com >
2026-06-02 10:56:44 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Wentao Ye and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
8f87eb4622
[Refactor] Clean up log once scope="local" ( #40540 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-22 16:42:43 -04:00
Michael Goin and Claude
eb4205fee5
[UX] Integrate DeepGEMM into vLLM wheel via CMake ( #37980 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-08 18:56:32 -07:00
Maral
2e9034c998
[W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. ( #33892 )
...
Signed-off-by: maral <maralbahari.98@gmail.com >
Signed-off-by: Maral <maralbahari.98@gmail.com >
2026-04-09 08:50:39 +08:00
Vadim Gimpelson
52069012fe
[Bugfix] Fix DeepGemm E8M0 accuracy degradation for Qwen3.5 FP8 on Blackwell ( #38083 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
2026-03-26 01:21:47 -07:00
Chauncey
09c3dc9186
[Revert] Remove CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function ( #37968 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-03-25 06:19:37 +00:00
SherryC41 and sherryC41
6ec5e9fd37
refactor: abstract deepgemm support into platform ( #37519 )
...
Co-authored-by: sherryC41 <sherry.c.c41@gmail.com >
2026-03-20 17:54:08 +00:00
Chauncey
7e08c22b8c
[Feat] Add CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function ( #35271 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-02-28 10:12:00 +00:00
ElizaWszola and Luka Govedič
a88b3be7c4
[Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales ( #33255 )
...
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-02-17 23:35:04 -08:00
Wentao Ye
67a746e87f
[Log] Optimize duplicate startup log ( #33944 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-02-06 17:49:56 +00:00
Xin Yang
79028d4388
[Perf] Disable clean_logits in deepgemm fp8_mqa_logits kernel ( #33568 )
2026-02-05 20:34:00 -05:00
Xin Yang
d08b356ee0
[Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper ( #32619 )
...
Signed-off-by: Xin Yang <xyangx@amazon.com >
2026-01-22 15:47:04 -05:00
Chauncey
c4e5bdf61b
[Bugfix] Fix the fp8_mqa_logits dim mismatch ( #32652 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-01-20 18:48:07 +08:00
Kevin McKay and Claude Sonnet 4.5
4614c5a539
[Bugfix][Hardware][AMD] Consolidate FP8 min/max values helper function ( #31106 )
...
Signed-off-by: c0de128 <kevin.mckay@outlook.com >
Signed-off-by: Kevin McKay <kevin@example.com >
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-07 06:55:03 +00:00
Kevin McKay
ec58c10ce1
[Misc] Fix quantization-related typos ( #31116 )
...
Signed-off-by: c0de128 <kevin.mckay@outlook.com >
2025-12-21 21:13:48 -08:00
Wentao Ye
3bd8335bd0
[Refactor] Refactor for DeepGemmQuantScaleFMT using cache ( #30898 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2025-12-19 13:50:39 -07:00
ElizaWszola
994acec0cc
[Bugfix] Fix fusion for VL models ( #30244 )
...
Signed-off-by: ElizaWszola <ewszola@redhat.com >
2025-12-14 21:22:37 +08:00
Roberto L. Castro and youkaichao
4fa7ce46f3
[Feature] Add SM103 (Blackwell Ultra) Support to vLLM ( #30484 )
...
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2025-12-12 19:34:23 -08:00
Zhewen Li
ae339b1a67
[Bugfix] Fix DeepGEMM after #29546 ( #30267 )
...
Signed-off-by: zhewenli <zhewenli@meta.com >
Signed-off-by: Zhewen Li <zhewenli@meta.com >
2025-12-09 01:05:27 +00:00
ElizaWszola and yewentao256
af0444bf40
[Performance] Fused blockwise quant RMS norm ( #27883 )
...
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: yewentao256 <zhyanwentao@126.com >
2025-12-07 16:38:04 +00:00
Pleaplusone
06c20c9904
[ROCm] Add AMD GPU support on Deepseek v3.2 and SparseMLA ( #26670 )
...
Signed-off-by: ganyi <ygan@amd.com >
2025-11-20 02:54:01 -08:00
Alexander Matveev and mgoin
3aaa94ac99
[Performance] Reduce DeepGEMM N dim restriction from 128 to 64 multiplier ( #28687 )
...
Signed-off-by: Alexander Matveev <amatveev@redhat.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2025-11-19 15:47:13 -08:00
Varun Sundar Rabindranath and Varun Sundar Rabindranath
fe1cd7704d
[Performance][B200] silu_mul_quant: pack scales in int32 ( #28358 )
...
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
2025-11-13 10:16:55 -08:00
Varun Sundar Rabindranath and Varun Sundar Rabindranath
74a9a9faad
[Performance][B200] Fix deepgemm prologue ( #27897 )
...
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
2025-11-12 13:13:03 -08:00
Yeshwanth N
71b1c8b667
[Chore]:Extract math and argparse utilities to separate modules ( #27188 )
...
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com >
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com >
Signed-off-by: yeshsurya <yeshsurya@gmail.com >
2025-10-26 04:03:32 -07:00
3ae082c373
[Chore] Separate out optional dependency checks from vllm.utils ( #27207 )
...
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2025-10-22 10:44:21 -04:00
Harry Mellor
6c9fdbf725
[Docs] Replace rst style double-backtick with md single-backtick ( #27091 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-17 02:47:34 -07:00