MOVZX and Hinata
61fbe0b865
[Mini-VLLM] Remove unreferenced definitions found by static scan
...
A symbol scan over every tracked file found 123 top-level definitions in
vllm/ with zero references, including references inside strings. All of
them came from features this fork deleted earlier.
- Delete dead attention helpers: MLADCPManager, split_prefill_chunks, the
reshape helpers, compute_varlen_chunk_metadata, the LINEAR backend's
unused BailingLinearAttentionBackend, and the sparse flash-attn entry
points.
- Delete the CPU and AMX op wrappers from _custom_ops.py: cpu_mla_decode,
cpu_mla_extend, cpu_mla_get_scheduler_metadata,
gather_mla_context_cache_cpu.
- Delete dead quantization helpers: awq_pack, gptq_pack,
gptq_quantize_weights, get_and_maybe_dequant_weights,
marlin_quant_fp8_torch, the rand_marlin_weight_*_like pair,
ocp_mx_weight_dtype_and_rows, get_fp8_block_weight_scale,
input_to_float8.
- Delete dead model helpers: resolve_visual_encoder_outputs,
run_dp_sharded_vision_model, fast_topk, get_layer_index,
is_pp_missing_parameter, skip_spec_layers, KimiK3MergedQKVGateLinear,
selective_scan_fn, causal_conv1d_update_torch.
- Delete dead utils: format_kib, format_mib, chunk_list, prev_power_of_2,
split_host_port, join_host_port, make_zmq_path, make_async_with_semaphore,
is_meta_module, is_non_overlapping_and_dense, supports_xpu_graph,
create_kv_caches_with_random*, and the unused has_* capability probes.
- Delete the unreachable deep-gemm mega-MoE warmup path, which imported
vllm.models.deepseek_v4.nvidia.model and vllm.models.kimi_k3.nvidia.model,
neither of which exists anymore.
- Drop the imports and the two now-empty guards left behind.
Verified statically: every changed file compiles, no unused import
remains, and no deleted name is referenced in vllm, tests or csrc. The
runtime gates still need the Python environment rebuilt.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:39:56 +07:00
MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
837e87e1c5
[Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts ( #61979 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:17:56 +08:00
68ebea789a
[Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts ( #59044 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 16:20:52 -04:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Rohan Potdar and Claude Opus 5.5
105605b8e9
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE ( #60282 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 14:10:12 -05:00
Chaitanya Sri Krishna Lolla and Cursor
833665cded
[ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe ( #59596 )
...
Signed-off-by: lcskrishna <lollachaitanya@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 23:20:14 -05:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
2308b6f00b
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE ( #58897 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-14 10:48:05 +08:00
Anirudh Ravichandran
e90158d912
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE ( #60733 )
2026-10-13 17:02:17 -04:00
Elvir Crnčević
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch ( #53911 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-10-13 11:10:38 -04:00
Divakar Verma and Claude Sonnet 5
f7ba007b5e
[ROCm][MoE] Split AiterExperts by weight-quantization key ( #60322 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-13 00:13:33 -05:00
deedc7be00
[Perf][Humming] Skip the redundant MoE input copy before the w13 GEMM ( #59340 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-12 20:24:12 +00:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
Ilya Markov
b3ee560ae8
[Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel ( #59042 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:02:56 -04:00
Wentao Ye
3709632ff2
[Refactor] Remove dead code multiple places ( #60120 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-10 15:41:10 +00:00
linitra24
364bc81a9e
[Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats ( #60812 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-10-10 12:52:51 +08:00
YiSheng5 and Kunshang Ji
0cc0460870
[XPU]Fix the accuracy issue when meet topk_ids=-1 on DP+EP scenarios ( #57787 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-09 20:01:54 +08:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
5ab4e4445b
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts ( #59128 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
40c86ae1ce
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x ( #58877 )
...
Signed-off-by: lazymio <mio@lazym.io >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 01:49:13 -04:00
Mason and Yongye Zhu
6dd813e0b7
[Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend ( #59571 )
...
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-10-05 16:24:14 -07:00
Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
4c4003bb5f
[Bugfix] Bind routed-experts capture to the MoE layer, not the kernel ( #59455 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 15:50:38 +00:00
5addde2a0a
[CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE ( #59434 )
...
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-01 15:51:53 +08:00
e12291d733
[CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers ( #54024 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 15:18:25 +08:00
0103a9b96f
[Quantization] Add per-token NVFP4 CuTe-DSL MoE backend ( #50030 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-09-30 10:42:03 +00:00
yhcheong (vllmellm)
16d4ac4ba3
[ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 ( #58262 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-30 17:23:00 +08:00
Fangzhou Ai and Cursor Agent
77e8b02c8d
[ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER ( #58819 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: Cursor Agent (Claude Sonnet 5) <cursoragent@cursor.com >
2026-09-30 15:07:44 +08:00
31afa4f1bb
[ROCm] Bump AITER to v0.1.23 ( #58867 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-27 19:04:58 -05:00
Hongxia Yang
3137ff0773
[ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 ( #58201 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-09-27 02:26:49 +00:00
e440b75bb6
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision ( #50605 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
2026-09-25 14:55:52 -05:00
Yongye Zhu and Claude Opus 5.5
378504a544
[MoE] Defer the TRTLLM-Gen top-k finalize on the modular path ( #58635 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-25 15:38:29 +08:00
3652f35e7c
[Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing ( #57954 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-24 18:36:08 -07:00
Misha Goin
cd06e81c41
[Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle ( #58427 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-24 12:37:33 -07:00
Shuolei Wang
9ed81581a8
[Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… ( #58051 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-09-23 21:03:21 +08:00
9127170b88
[Quantization] Select per-token NVFP4 MoE backends explicitly ( #57176 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 07:30:58 +00:00
Garrett Goon and Claude Opus 5
8bf051cd51
[Bugfix][Kernel] Skip the fused silu-mul block-quant fast path when a swiglu clamp is set ( #57984 )
...
Signed-off-by: Garrett Goon <garrett@primeintellect.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-21 22:38:38 +00:00
Andreas Karatzas and OpenAI Codex
12972ae40b
[Bugfix][ROCm] Reject unsupported EP for monolithic AITER MXFP4 MoE ( #57866 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 15:30:21 -05:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
Misha Goin and ErinYin
e5fce7b56b
[Core][Kernel] Share persistent workspaces for Marlin and Humming ( #57421 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: ErinYin <liang.yin@daocloud.io >
2026-09-20 17:28:33 +08:00
1cdf1689e5
[Quantization][XPU] Enable int8_w8a8 MoE on the Triton backend for XPU ( #53162 )
...
Signed-off-by: Artur Fierka <artur.fierka@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-18 22:38:15 +08:00
Kevin Li and kevinli
911548247a
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend ( #52101 )
...
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
2026-09-18 10:34:01 -04:00
Yuwen Zhou and Li, Jiang <jiang1.li@intel.com>
b346479065
[CPU] Add CPU FP8 W8A8 linear/MoE support ( #49942 )
...
Signed-off-by: yuwenzho <yuwen.zhou@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-18 17:59:42 +08:00
meng-amd
6df2b1a8c6
[Quantization] Support native Quark W4A16 INT4/UINT4 exports in vLLM ( #48606 )
...
Signed-off-by: HongWei Meng <HongWei.Meng@amd.com >
2026-09-18 17:52:46 +08:00
Rohan Potdar and Claude
213c379fca
[ROCm][Bugfix] Gate AITER MXFP8 MoE on the aiter enable flag ( #57426 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-18 00:06:05 +00:00
Misha Goin
ac78445bc9
[MoE] Encapsulate TRT-LLM BF16 weight layout handling ( #57405 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-17 17:47:30 -04:00
117cf43fb6
[HARDWARE][POWER] Enable W8A8 INT8 MoE on POWER ( #55316 )
...
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com >
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com >
Co-authored-by: Antigravity <antigravity@google.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-17 21:45:13 +08:00
Lucas Wilkinson and Robert Shaw
fe284ea17d
[BugFxi] Fix DeepGEMM FP8 workspace over allocation ( #53914 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-17 09:35:34 -04:00