2067297b2f
[ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var ( #61278 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Signed-off-by: afriedri <afriedri@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-17 04:00:28 -05:00
28e92eeff2
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill ( #59552 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 22:58:56 +00:00
cc2e4fecf2
[ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm ( #52831 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 15:19:07 -05:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
3f1e57225a
[ROCm][Perf][Kimi-K3] Use AITER Gluon KDA kernels for decode and spec decode ( #57392 )
...
Signed-off-by: omuhamma <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-15 17:54:24 -05:00
e711723352
[ROCm] gfx1250 optimizations and model enablement ( #58800 )
...
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: Juan Villamizar <jpvillam@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
2026-10-15 14:51:57 -07:00
Mikko Tukiainen
0c25849292
[ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K ( #59406 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
2026-10-15 21:11:33 +00:00
Mikko Tukiainen and Cursor
47fd76e4ce
[ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating ( #56226 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-15 17:53:18 +00:00
f9b65c30c7
[ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 ( #60645 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 10:11:57 +00:00
c6566bb9f6
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather ( #59070 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 09:20:38 +08:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
Mikko Tukiainen and Cursor
548597367e
[ROCm][Perf] Add AITER FlyDSL GDN prefill backend ( #57560 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 09:44:22 -05:00
eky-amd and Claude Fable 5
29f955cce0
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 ( #54857 )
...
Signed-off-by: Ethan Ky <ethan.ky@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-06 11:26:38 -05:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
98b29aa99a
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes ( #58167 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
Fangzhou Ai and Cursor Agent
77e8b02c8d
[ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER ( #58819 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: Cursor Agent (Claude Sonnet 5) <cursoragent@cursor.com >
2026-09-30 15:07:44 +08:00
Muhammad Ahmed
0af34418e9
[ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel ( #58655 )
...
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com >
2026-09-29 10:19:10 +08:00
Mehmet Cagri
70ae0b7435
[ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter ( #53492 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com >
2026-09-29 09:08:18 +08:00
Hongxia Yang
3137ff0773
[ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/a16w4 ( #58201 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-09-27 02:26:49 +00:00
Divakar Verma
65c5e24689
[ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix ( #58393 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-09-24 21:14:02 -05:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
1c0eee919d
[ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness ( #51915 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
2026-09-22 18:08:51 -05:00
874a6b5a06
[ROCm] Refactor tuned gemms ( #55001 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Signed-off-by: afriedri <afriedri@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-21 16:30:16 +00:00
b8cf275382
[AMD][Minimax-M3][perf] Enable packed LBHNC AITER QK-norm fusion for MiniMax-M3 on ROCm ( #54535 )
...
Signed-off-by: Wei-Ting Liao <wei-ting.liao@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-21 16:38:30 +08:00
afriedri
a9a7e45f31
[ROCm] Restore VLLM_ROCM_USE_AITER_FP4_ASM_GEMM and default w4a4 ASM GEMM back to off ( #57055 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
2026-09-17 13:39:52 -05:00
1df336c3ba
[Perf][ROCm] Enable AITER QuickReduce + RMSNorm fusion ( #48249 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-17 15:05:48 +00:00
Shanshan Shen
bc0f47cd03
[ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 ( #57132 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-16 18:58:23 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
a4d2d9d95e
[ROCm][Kimi-K3] Enable a4w4 flydsl kernels for KimiK3 ( #53940 )
...
Signed-off-by: ppalanga <ppalanga@amd.com >
Signed-off-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com >
2026-09-16 01:38:02 +00:00
35dc273072
[ROCm][Spec Decode] Add Aiter MLA decode support non-causal draft block ( #55966 )
...
Signed-off-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com >
Co-authored-by: Poovaiah Palangappa <poovaiah.palangappa@amd.com >
Co-authored-by: John Qin <zhengsheng1997@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com >
2026-09-15 12:18:09 -05:00
Fangzhou Ai and Cursor
00972dfd72
[ROCm][DSV4.1][Perf] Fold the mHC post step into the delayed pre projection ( #56513 )
...
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai >
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-14 16:44:56 -05:00
Fangzhou Ai and Fangzhou Ai
46d2b23ac5
[ROCm][DSV4.1][Perf] Use AITER mHC for the delayed pre block ( #56503 )
...
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai >
Co-authored-by: Fangzhou Ai <fangzhou@semianalysis.ai >
2026-09-12 00:35:48 +00:00
e77daef89e
[Model] Support DeepSeek-V4.1-Flash ( #56214 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 02:11:19 -07:00
Simon Danielsson
7470082f57
[ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM ( #51692 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-10 16:18:37 +00:00
Taoyu Zhu and Shanshan Shen
ce6c241ddc
[ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path ( #54787 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-08 07:21:49 +00:00
Andreas Karatzas and OpenAI Codex
e476556189
[ROCm][CI] Add attention-sink support to ROCm AITER sparse MLA ( #54404 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 09:26:47 +00:00
de69e821b7
[ROCm][Perf][DeepSeek V4] Fuse native FP8 shared expert with MXFP4 routed experts ( #53161 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Cursor Agent <agent@cursor.com >
2026-09-06 19:42:21 -07:00
Jee Jee Li
199cb9b964
[Kernel] Remove unused fake implementation ( #55535 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-06 17:54:32 -07:00
Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
Harry Mellor and Claude Opus 5
d539de1c5d
[Docs] Add missing return annotations flagged by griffe ( #54980 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-02 09:56:07 -07:00
afriedri and Andreas Karatzas
2f1cba799e
[ROCm] remove VLLM_ROCM_USE_AITER_FP4_ASM_GEMM environment variable; make w4a4 use the preshuffle triton+asm by default ( #53141 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Signed-off-by: afriedri <afriedri@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-28 18:59:53 +00:00
Shanshan Shen
32ad1400d7
[ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 ( #53540 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-08-27 09:37:54 -07:00
Shanshan Shen
d626108b18
[ROCm][Perf] Fuse DeepSeek-V4 mHC post/pre and RMSNorm with AITER ( #52737 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-08-20 04:51:04 +00:00
Divakar Verma
49fb2ee348
[ROCm][CI] add Aiter ops tests ( #52208 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-08-17 18:33:22 -05:00
PikaPikachu
466855a2bf
[ROCm] Enable DeepSeek-V4 on gfx11 ( #47017 )
...
This PR enables DeepSeek-V4 checkpoints on ROCm gfx11/RDNA devices.
Signed-off-by: Letian Kang <Letian.Kang@amd.com >
2026-08-12 08:55:10 +08:00
Andreas Karatzas and OpenAI Codex
1ab2801dde
[ROCm] Remove stale SDPA and skinny GEMM workarounds ( #50907 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 14:19:16 -04:00
skysnow2001 and Douglas Lehr
f43e1d26e3
[ROCm] Enable AITER and FP8 inference on GFX120x ( #43615 )
...
Signed-off-by: skysnow2001 <skysnow9285@gmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-08-03 16:45:37 -05:00
Hongxia Yang
8f50685c48
[ROCm][Kimi-K3] aiter moe environment variable cleanup ( #50582 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-08-03 18:54:04 +00:00