Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
Julien Debache
ae934ba8a5
feat: extended EPLB support for Mistral Large 3 and additional MoE backends ( #48355 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-07 05:32:52 -07:00
pmanczak and pmanczak
6b5bec7bed
[Misc] Add and enable Triton kernel unit tests on XPU ( #45694 )
...
Signed-off-by: pmanczak <pmanczak@users.noreply.github.com >
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: pmanczak <pmanczak@users.noreply.github.com >
2026-08-07 02:53:03 -07:00
0de0362ea1
[ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes ( #48847 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-07 01:19:26 -07:00
e08111211b
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation ( #47106 )
...
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-06 17:42:45 -07:00
d8eabdbfbe
[ROCm][MLA] Use asm decode for non-divisor small head counts ( #50578 )
...
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-08-06 14:56:53 -04:00
b38e111d3e
[Attention][MLA] Per-request scheduling for MLA chunked context ( #50613 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 16:45:44 +00:00
zofia and Kunshang Ji
8060260e1d
[Linear] [Kernel] add block-wise scaled_mm ( #49932 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 14:40:26 +08:00
Gabriel Wu and Yongye Zhu
2e35c529b8
[Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS ( #51038 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-05 23:19:22 -07:00
Aarushi Jain
71f975af43
[ROCm][CI] Add MLA decode accuracy and determinism tests ( #50480 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
2026-08-05 20:35:56 -05:00
Rohan Potdar and Claude Opus 4.8
38ebd97bca
[ROCm] Relax MLA rope+cache test tolerances for bf16 ( #51083 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-05 17:50:52 -05:00
Micah Williamson and Andreas Karatzas
4282fe52d5
[ROCm][CI] Add More AITER quantization/MoE kernel tests ( #49375 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 21:27:33 +00:00
stefankoncarevic and Andreas Karatzas
65addac701
[ROCm][CI] Keep rocprofiler-sdk out of DeepEP HT MoE test workers ( #51173 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 15:28:41 -05:00
Rohan Potdar and Claude Opus 4.8
c2d8009044
[ROCm] Work around DeepEP teardown SIGSEGV in MoE test harness ( #51174 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-05 14:44:42 -05:00
Michael Goin and OpenAI Codex
8f158d0ee2
[MoE] Share apply_moe_activation support metadata ( #44359 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-05 12:05:47 -07:00
Gabriel Wu and Pavani Majety
08b8613b7b
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness ( #48929 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Pavani Majety <pmajety@nvidia.com >
2026-08-05 07:25:46 -07:00
amitz-nv and OpenAI Codex
52c0e3cb08
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection ( #50157 )
...
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-04 09:40:23 -07:00
c810937573
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 ( #49791 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-03 16:49:41 -07:00
aoshen02 and Nick Hill
42ab184ea7
[MRV2] Enable routed-experts capture ( #50721 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-03 21:00:13 +00:00
Andreas Karatzas
e279f71583
[ROCm][Test] Fix AITER MXFP4 oracle contract ( #50728 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-03 20:53:18 +00:00
0a6446005d
[CPU] Migrate unquantized MoE to the modular-kernel experts structure ( #50133 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-08-03 14:55:04 +08:00
Jason Li
0055b8bfa3
[Attention][MiniMax-M3] Add MSA speculative decode verification ( #50032 )
2026-08-02 13:46:55 -07:00
Matt Qin and yudigege86
77469c9057
[ROCm][MLA] Mask the AITER MLA small-head verify flatten causally ( #50476 )
...
Signed-off-by: yudigege86 <naqin@amd.com >
Co-authored-by: yudigege86 <naqin@amd.com >
2026-08-01 07:17:17 +00:00
Amir Samani
c4a4a42d80
[Bugfix][Test] Fix monolithic routing replay test buffer capacity ( #50640 )
2026-07-31 19:37:39 -07:00
b40d859c7b
[Kernel][Helion] Add numerics checks to benchmark script ( #48968 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-31 20:16:02 -04:00
454ea5b526
[MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod ( #44570 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com >
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com >
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com >
Co-authored-by: Tushar Jain <8455015+tushar00jain@users.noreply.github.com >
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-07-31 14:13:03 -07:00
fxmarty-amd
e67a2e0a56
[ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend ( #48949 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-07-31 10:07:15 -07:00
94e9ef0768
[Bugfix] Don't transpose fused MoE quantization scales in RoutedExperts.load_weights ( #50137 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:46:52 -07:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
df71917cf1
[DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. ( #49236 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
Chaojun Zhang and Kunshang Ji
482cfc20f0
[XPU] Unify XPU RMSNorm kernels with vllm_c and drop redundant XPU-specific implementation ( #46981 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-31 17:28:18 +08:00
Thien Tran and Codex
34bb795ff3
[CI] Add M3 MSA tests to CI ( #49143 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <noreply@openai.com >
2026-07-31 01:01:48 -07:00
10e6b40015
[CPU][BugFix] Remove redundant kv cache write ( #50437 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 15:28:44 +08:00
stefankoncarevic and Andreas Karatzas
2773ec31b3
[ROCm][CI] Use explicit wvSplitKrc skinny-GEMM test tolerance for bf16 (gfx950) ( #49309 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 22:25:42 -05:00
Matt and Andreas Karatzas
d91f7af7d0
[Hardware][AMD][Kernel][CI][Bugfix] Fix ROCm DeepEP FP8 max ( #50467 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 21:37:36 -05:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
Wentao Ye
0bff0ce5d3
[Kimi K3 Bug] Fix deepgemm support for kimi k3 ( #50458 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 20:10:16 -04:00
Shangdi Yu and OpenAI Codex
45b60e3914
[Kernel][Helion] Disable unsafe B200 RMS reduction warp specialization ( #50345 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 20:37:52 +00:00
Michael Goin
0eec856cc3
Add Humming indexed-MoE regression test ( #50468 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-07-30 16:26:24 -04:00
Andreas Karatzas
bdc98bf506
[CI] Initialize fused gated RMSNorm weights ( #50377 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 15:34:08 -04:00
Wentao Ye
837eae6458
[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement ( #50298 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:39:07 -04:00
Andreas Karatzas
5b95890742
[FlexAttention] Avoid encoder block-mask compile explosion ( #50339 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 11:24:13 -04:00
e2efe79695
[ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ ( #47207 )
...
Signed-off-by: sroberts-amd <sroberts@amd.com >
Signed-off-by: Stacy Roberts <sroberts@amd.com >
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 08:37:43 -05:00
Martin Hickey and Harry Mellor
38a267cdd5
[MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent ( #49570 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-30 05:02:51 -07:00
Li, Jiang and Claude
072a472782
[CPU] Bump up CPU kernels to latest version ( #50387 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-30 19:18:43 +08:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Andreas Karatzas and OpenAI Codex
0c64be8873
[Test][ROCm] Account for gfx950 FP8 RMSNorm rounding ( #49839 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 18:46:37 +08:00
yinfengLiu
f1e8fd27ae
[ROCm] Add AITER FP8 ViT encoder attention ( #49937 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-07-30 06:22:10 +00:00
Jane (Yuan) Xu
a7a204cc6e
Add FlashMLA H100 tests to CI, fix them after #32810 ( #50322 )
2026-07-29 22:06:18 -07:00
stefankoncarevic and Andreas Karatzas
b88916617d
[ROCm] [CI] Support cached K/V (key/value=None) in Triton prefix-prefill ( #48257 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-29 23:31:51 -05:00
