MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
Kevin H. Luu and Claude Opus 5.5
a956a8126e
[Bugfix][MoE] Keep the gate's hash_indices_table out of RoutedExperts parameters ( #61930 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 18:00:17 -07:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Rohan Potdar and Claude Opus 5.5
105605b8e9
[ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE ( #60282 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 14:10:12 -05:00
189bbfb9f4
Add MoNe experts pruning support ( #50415 )
...
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:18:43 +00:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
roikoren755 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
51eeb0c58f
[Bugfix] Load stacked expert weights for non-gated MoE ( #59031 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 15:26:55 +03:00
Andreas Karatzas and OpenAI Codex
9179bcd416
[Bugfix][MoE] Reject hash routing for unsupported monolithic backends ( #57867 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 16:28:35 -04:00
ac2f0ea82c
[MoE][Bugfix] Skip SP padded rows in grouped MoE routing ( #56079 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-17 19:36:19 +00:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Itay Alroy
658c8131c7
[Elastic EP] Reuse CUDA graphs across reconfiguration ( #54985 )
2026-09-12 10:26:39 -04:00
Isotr0py
1356635d83
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support ( #54566 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-02 11:09:09 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
Andreas Karatzas and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
36f4630d89
[Bugfix][MoE] Fix fused block-scale orientation ( #50727 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 16:05:38 -04:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
+12
7c6729b769
[Model] Add Kimi K3 support: model files and kernels [1/N] ( #50089 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-07-29 14:10:58 +08:00
Martin Hickey and Harry Mellor
6cf7b26bd4
[docs] Fix the docs build ( #48008 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-08 15:47:22 -07:00
Wei Zhao and Yongye Zhu
2afa3f7e95
[Perf] Minimax M3 - Support cross-layer allreduce-norm fusion ( #47631 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-07-07 21:16:32 -07:00
Harry Mellor
1ab9522935
Remove more unnecessary load_weights methods ( #47058 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-30 15:22:16 +01:00
Hongxia Yang and Claude Opus 4.8
c2507fb293
[ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model ( #46545 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-06-26 07:05:20 -07:00
qli88 and TJian
8e394244a5
[ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4 ( #46419 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-26 06:35:35 -07:00
0a1c5034f5
[Model] Add MiniMax M3 support ( #45381 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-16 01:01:25 +08:00
bnellnm
29026682cb
[Bugfix] Fix nemotron accuracy drop introduced by #41184 ( #45037 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-10 13:16:25 -04:00
bnellnm
dc68bd8c41
[MoE Refactor] FusedMoE/MoERunner inversion refactor ( #41184 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-08 10:42:58 -04:00
JartX and TJian
062b05ff3a
[ROCm][Perf] Fused MoE W4A16 HIP kernel for AMD RDNA3 (gfx1100) ( #44075 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-06-06 15:30:39 -05:00
aoshen02 and Claude Opus 4.8
f87df1df9e
[Bugfix][MoE] Snapshot max_cudagraph_capture_size into FusedMoEConfig ( #44613 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-05 23:14:41 -07:00
Harry Mellor
a80af24356
Speed up docs build ( #44635 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-05 14:51:44 +00:00
MHYangAMD and MHYangAMD
ab12aab127
[Bugfix] [ROCm] [DSV4] Fix AITER MXFP4 MoE weight loading and shuffle… ( #42595 )
...
Co-authored-by: MHYangAMD <MHYangAMD@users.noreply.github.com >
2026-05-29 04:08:33 -07:00
b690b2bb67
[Model]Support Step-3.7-Flash ( #43859 )
...
Signed-off-by: luotingdan <luotingdan@stepfun.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: luotingdan <luotingdan@stepfun.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Yu Huang <yuhuang@nvidia.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-05-28 17:01:48 -07:00
Yongye Zhu
2d2c660104
[MoE] Remove inplace fused experts mechanism ( #43727 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-27 20:00:19 -07:00
sychen52
fb21d8b4f9
Add NVFP4 MOE support for Deepseek V4. ( #42209 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-05-22 07:21:51 -07:00
fa2a33b893
[Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py ( #38288 )
...
Signed-off-by: Chengyi Nie <cnie@roblox.com >
Co-authored-by: Chengyi Nie <cnie@roblox.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-15 08:25:52 +08:00
aoshen02
8c79ad6580
Revert "[Core] Replace routing replay with device cache and async D2H pipeline" ( #39917 ) ( #42434 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-05-13 23:49:01 -07:00
bnellnm and Robert Shaw
d9b4990783
[MoE Refactor] EPLB refactoring for FusedMoE ( #41055 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-05-12 14:16:31 -04:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
206eaed08d
[MoE Refactor] Move expert map related code into ExpertMapManager class ( #41046 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 09:18:27 -04:00
1b57eb41f2
[MoE] Move various experts classes to fused_moe/experts/ ( #41979 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
2026-05-11 07:54:33 +08:00
2c6b59b807
[ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next ( #39280 )
...
Signed-off-by: nholmber <nholmber@users.noreply.github.com >
Signed-off-by: Tres Popp <tres.popp@amd.com >
Signed-off-by: Doug Lehr <douglehr@amd.com >
Co-authored-by: nholmber <nholmber@users.noreply.github.com >
Co-authored-by: Tres <tpopp@users.noreply.github.com >
Co-authored-by: Tres Popp <tres.popp@amd.com >
Co-authored-by: Doug Lehr <douglehr@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-05-08 15:38:00 -04:00
TomerBN-Nvidia
8189a15914
[Core] Replace routing replay with device cache and async D2H pipeline ( #39917 )
...
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com >
2026-05-07 11:24:56 -07:00
Chendi.Xue and Kunshang Ji
685bf811d6
[XPU] enable is_act_and_mul for xpu ( #37481 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-05-05 01:07:39 +00:00
Jinzhen Lin
08834cc3ce
[Quantization] add humming mxfp4 moe backend ( #41083 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
2026-05-02 18:36:03 -07:00
b55b26520c
[MoE] Make MoERunnerInterface a PluggableLayer for OOT support ( #35178 )
...
Signed-off-by: wxsIcey <1790571317@qq.com >
Signed-off-by: Icey <1790571317@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-30 03:31:08 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Jinzhen Lin
9f771b3ab9
[Quantization] add humming quantization kernel ( #34556 )
2026-04-24 09:29:44 -04:00
Itay Alroy
c9d3c6e6af
fused_moe: treat NIXL EP as batched experts ( #40412 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-04-24 08:05:31 -05:00
bnellnm
1c2c1eb8b9
[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping ( #40671 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-23 11:22:34 -04:00
bnellnm
809d83c2dc
[MoE Refactor] Combine MoERunnerBase + DefaultMoERunner ( #40560 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-22 14:43:17 +00:00
bnellnm and Robert Shaw
726efe177b
[MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase ( #35949 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-20 12:28:46 -04:00
bnellnm
19ec9a0a62
[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework ( #35549 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-14 16:11:20 -04:00
Hexiang Wang
f02b3269e7
[PluggableLayer][3/N] Apply PluggableLayer to moe-related layers. ( #33556 )
...
Signed-off-by: whx-sjtu <2952154980@qq.com >
2026-04-14 09:55:00 -04:00