MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
4d58fa254a
[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 ( #61538 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 19:02:01 +00:00
simon-veitner-redhat and Claude
ae94e9c9e8
[Bugfix] Don't crash DeepGEMM MegaMoE warmup when get_bf16_mega_gate_config is missing ( #61657 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 18:58:35 +00:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving ( #60468 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
2026-10-13 20:17:48 +08:00
Chris Leonard and Shengqi Chen
cc73cca9c8
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) ( #48962 )
...
Signed-off-by: Chris Leonard <chleonar@redhat.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 08:39:04 -07:00
Lucas Wilkinson
bd2d7ae7c1
[Perf] Reuse MoE workspace for DeepGEMM warmup ( #57268 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-17 03:01:19 -04:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Ayushman Singh and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7bcca16729
[Bugfix] Fix DeepGEMM FP8 warmup coverage ( #56452 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-12 14:20:38 -04:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Michael Goin
917fdb5bf7
[Bugfix] Fix DeepGEMM warmup when using FlashInferFp8DeepGEMMDynamicBlockScaledKernel ( #49467 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-07-22 16:28:49 -07:00
ap9272
cac3e70cd4
Correct model layer aliasing for Bert style models ( #43896 )
2026-07-09 19:46:22 -04:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
bnellnm
dc68bd8c41
[MoE Refactor] FusedMoE/MoERunner inversion refactor ( #41184 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-08 10:42:58 -04:00
bnellnm
6427603ae8
[MoE Refactor] Move remaining experts classes to experts directory ( #42334 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 09:19:46 -04:00
Vasiliy Kuznetsov
38fa87caca
mxfp8 online quant move to new frontend ( #40152 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-20 06:26:12 -07:00
a776a48b1c
[MoE] Move DEEP_GEMM into experts/ subdirectory ( #39005 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-08 19:23:08 +00:00
Wei Zhao and Robert Shaw
1af6f78ae5
[Perf] Change Trtllm fp8 MoE to use Shuffled Weights and BlockMajorK Layout ( #38993 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-05 10:54:31 -04:00
EdalatiAli and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
daa05bf340
[Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed ( #37358 )
...
Signed-off-by: EdalatiAli <aliedalati@cohere.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-19 17:58:33 +00:00
Xinan Miao and SouthWest7
2cdf92228c
[Feature]: Remove Chunking From FusedMoE ( #34086 )
...
Signed-off-by: SouthWest7 <am1ao@qq.com >
Signed-off-by: Southwest <1403572259@qq.com >
Signed-off-by: southwest <am1ao@qq.com >
Signed-off-by: Xinan Miao <1403572259@qq.com >
Co-authored-by: SouthWest7 <am1ao@qq.com >
2026-03-12 14:24:38 -04:00
97995f6376
[MoE Refactor] Create MK for TRTLLM Kernels ( #32564 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com >
2026-03-03 10:39:50 -08:00
Kunshang Ji
8ad54a991b
[Platform] Add current_platform.num_compute_units interface ( #35042 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
2026-02-24 22:22:49 -08:00
emricksini-h
325ab6b0a8
[Feature] OTEL tracing during loading ( #31162 )
2026-02-05 16:59:28 -08:00
5a93b9162b
[MoE Refactor] Integrate Naive Prepare Finalize into MK ( #32567 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com >
2026-01-27 01:28:02 +00:00
Robert Shaw
42135d6898
[MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority ( #32414 )
2026-01-21 08:22:33 -05:00
tjp_zju and Wentao Ye
2f03035a61
"refactor: refactor_repeated_interfaces" ( #32486 )
...
Signed-off-by: tom-zju <tanjianpingzju1990@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-01-18 22:07:01 +08:00
Matthew Bonanni
4a8412f773
[UX] Reduce DeepGEMM warmup log output to single progress bar ( #30903 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2025-12-17 20:21:51 -08:00
Lucas Wilkinson
aacf0abf8b
[BugFix] Fix AttributeError: 'MergedColumnParallelLinear' object has no attribute 'weight_scale' ( #30399 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2025-12-10 07:59:23 -08:00
Wentao Ye
de540c0354
[Feature] Add env var VLLM_MOE_USE_DEEP_GEMM ( #28422 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2025-11-11 02:29:48 +00:00
bnellnm
938772af03
[Kernels] Isolate modular kernel code from FusedMoEMethodBase subclasses. ( #27123 )
2025-11-04 21:59:45 +08:00
Wentao Ye and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
b3dda72c23
[Feature] Migrate DeepGEMM API from get_m_alignment_for_contiguous_layout to get_mk_alignment_for_contiguous_layout ( #26935 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-10-16 16:46:48 -04:00
0d21b9b51e
[UX] Speedup DeepGEMM warmup with heuristics ( #25619 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
2025-10-13 07:59:27 -07:00
Harry Mellor
d6953beb91
Convert formatting to use ruff instead of yapf + isort ( #26247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-05 07:06:22 -07:00
Wentao Ye and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
88d7bdbd23
[Bug] Fix AttributeError: 'FusedMoE' object has no attribute 'w13_weight_scale'. Did you mean: 'w13_weight_scale_inv' ( #25519 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-09-24 00:07:51 +00:00
bnellnm and Michael Goin
f11e3c516b
[Kernels] Support blocked fp8 quantization for compressed tensors MoE ( #25219 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2025-09-23 16:11:34 +00:00
Lucas Wilkinson
9fac6aa30b
[BugFix] Fix DeepGEMM warmup, no m.weight_scale_inv ( #25206 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2025-09-18 14:26:28 -07:00
bnellnm
5963b98b46
[Kernel] Delegate construction of FusedMoEQuantConfig to FusedMoEMethodBase subclasses ( #22537 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2025-09-17 17:43:31 -06:00
Lucas Wilkinson
2e6bc46821
[Startup] Make DeepGEMM warmup scale with max-num-batched-tokens ( #24693 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2025-09-11 20:10:19 -04:00
Varun Sundar Rabindranath and Varun Sundar Rabindranath
f703b923f3
[Misc] DeepGEMM : Avoid JIT generation in the hot-path ( #22215 )
...
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com >
2025-08-08 16:09:59 -07:00