275 Commits
Author SHA1 Message Date
837e87e1c5 [Feature][LoRA][MoE] Expert LoRA for FlashInfer TRTLLM MXFP4 experts (#61979)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 20:17:56 +08:00
68ebea789a [Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-10-16 16:20:52 -04:00
e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
2026-10-15 14:51:57 -07:00
Rohan PotdarandClaude Opus 5.5 105605b8e9 [ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:10:12 -05:00
Chaitanya Sri Krishna LollaandCursor 833665cded [ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe (#59596)
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-14 23:20:14 -05:00
fe98d90a9d [Dependency] Update Humming and remove NVFP4 scale workaround (#61556)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-14 13:32:36 -07:00
2308b6f00b [Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE (#58897)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-10-14 10:48:05 +08:00
Anirudh Ravichandran e90158d912 [Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE (#60733) 2026-10-13 17:02:17 -04:00
Elvir Crnčević b9b401b2ef [Kernel] Fuse DeepEP-v2 decode globalize+align into one launch (#53911)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
2026-10-13 11:10:38 -04:00
Divakar VermaandClaude Sonnet 5 f7ba007b5e [ROCm][MoE] Split AiterExperts by weight-quantization key (#60322)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-13 00:13:33 -05:00
deedc7be00 [Perf][Humming] Skip the redundant MoE input copy before the w13 GEMM (#59340)
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-12 20:24:12 +00:00
e0317b28ce [Feature] Support batch invariance on ROCm (#52231)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-12 10:53:26 -05:00
Ilya Markov b3ee560ae8 [Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel (#59042)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
2026-10-12 10:02:56 -04:00
Wentao Ye 3709632ff2 [Refactor] Remove dead code multiple places (#60120)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-10-10 15:41:10 +00:00
linitra24 364bc81a9e [Bugfix][LoRA] Fix GPT-OSS gate/up interleaving with mixed MoE LoRA formats (#60812)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
2026-10-10 12:52:51 +08:00
YiSheng5andKunshang Ji 0cc0460870 [XPU]Fix the accuracy issue when meet topk_ids=-1 on DP+EP scenarios (#57787)
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-09 20:01:54 +08:00
Misha GoinandClaude e3a29aa9b3 [Bugfix][MoE] Size the permute scratch from its inputs (#60447)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-08 15:32:07 -04:00
Tyler Michael SmithCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
5ab4e4445b [Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts (#59128)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
40c86ae1ce [Kernel] Enable OAI Triton MXFP4 MoE on SM12x (#58877)
Signed-off-by: lazymio <mio@lazym.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
2026-10-06 01:49:13 -04:00
MasonandYongye Zhu 6dd813e0b7 [Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend (#59571)
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-10-05 16:24:14 -07:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049)
Signed-off-by: jdebache <jdebache@nvidia.com>
2026-10-02 15:43:33 -04:00
4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 15:50:38 +00:00
5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434)
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-10-01 15:51:53 +08:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 15:18:25 +08:00
0103a9b96f [Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
2026-09-30 10:42:03 +00:00
yhcheong (vllmellm) 16d4ac4ba3 [ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 (#58262)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
2026-09-30 17:23:00 +08:00
Fangzhou AiandCursor Agent 77e8b02c8d [ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER (#58819)
Signed-off-by: fai <fangzhouai@gmail.com>
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Co-authored-by: Cursor Agent (Claude Sonnet 5) <cursoragent@cursor.com>
2026-09-30 15:07:44 +08:00
31afa4f1bb [ROCm] Bump AITER to v0.1.23 (#58867)
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-27 19:04:58 -05:00
Hongxia Yang 3137ff0773 [ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 (#58201)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
2026-09-27 02:26:49 +00:00
e440b75bb6 [ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: jpvillam <juan.villamizar@amd.com>
2026-09-25 14:55:52 -05:00
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 15:38:29 +08:00
3652f35e7c [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954)
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-09-24 18:36:08 -07:00
Misha Goin cd06e81c41 [Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-09-24 12:37:33 -07:00
Shuolei Wang 9ed81581a8 [Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… (#58051)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
2026-09-23 21:03:21 +08:00
9127170b88 [Quantization] Select per-token NVFP4 MoE backends explicitly (#57176)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 07:30:58 +00:00
Garrett GoonandClaude Opus 5 8bf051cd51 [Bugfix][Kernel] Skip the fused silu-mul block-quant fast path when a swiglu clamp is set (#57984)
Signed-off-by: Garrett Goon <garrett@primeintellect.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:38:38 +00:00
Andreas KaratzasandOpenAI Codex 12972ae40b [Bugfix][ROCm] Reject unsupported EP for monolithic AITER MXFP4 MoE (#57866)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-21 15:30:21 -05:00
7d99c2c4fd [Feature][Humming] Humming feature integration (#56685)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
2026-09-20 08:11:01 -07:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
2026-09-20 17:28:33 +08:00
1cdf1689e5 [Quantization][XPU] Enable int8_w8a8 MoE on the Triton backend for XPU (#53162)
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-18 22:38:15 +08:00
Kevin Liandkevinli 911548247a [Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101)
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
2026-09-18 10:34:01 -04:00
Yuwen ZhouandLi, Jiang <jiang1.li@intel.com> b346479065 [CPU] Add CPU FP8 W8A8 linear/MoE support (#49942)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-18 17:59:42 +08:00
meng-amd 6df2b1a8c6 [Quantization] Support native Quark W4A16 INT4/UINT4 exports in vLLM (#48606)
Signed-off-by: HongWei Meng <HongWei.Meng@amd.com>
2026-09-18 17:52:46 +08:00
Rohan PotdarandClaude 213c379fca [ROCm][Bugfix] Gate AITER MXFP8 MoE on the aiter enable flag (#57426)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-18 00:06:05 +00:00
Misha Goin ac78445bc9 [MoE] Encapsulate TRT-LLM BF16 weight layout handling (#57405)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-09-17 17:47:30 -04:00
Rukhaiya2004Akash kaothalkarAntigravityLi, Jiang <jiang1.li@intel.com>
117cf43fb6 [HARDWARE][POWER] Enable W8A8 INT8 MoE on POWER (#55316)
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-17 21:45:13 +08:00
Lucas WilkinsonandRobert Shaw fe284ea17d [BugFxi] Fix DeepGEMM FP8 workspace over allocation (#53914)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-09-17 09:35:34 -04:00
Qiming Zhang e6b1a5e5e3 [XPU] Enable XPU eplb (#44987)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
2026-09-17 13:19:13 +08:00
3e267bae70 [ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
2026-09-16 20:03:34 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00