cc2e4fecf2
[ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm ( #52831 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 15:19:07 -05:00
Jack Hu and Douglas Lehr
ef38e8d006
[Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN ( #42755 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jingxujaster@hotmail.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-15 17:10:14 -07:00
Mikko Tukiainen
abad32a718
[ROCm][Perf] Fix Gemma AR+RMS prefill fallback ( #58547 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com >
2026-10-12 19:53:08 +00:00
Woosuk Kwon
194bd72be7
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP ( #60517 )
2026-10-08 08:28:23 -07:00
eky-amd and Claude Fable 5
29f955cce0
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 ( #54857 )
...
Signed-off-by: Ethan Ky <ethan.ky@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-06 11:26:38 -05:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
834ad456d3
[Refactor] Remove dead tests code ( #58916 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 04:39:03 +00:00
5dd693122c
[MyPy][2/N] Fix mypy errors in small tests/ dirs ( #51043 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 21:27:45 +01:00
Laith Sakka
b39f760cc7
[torch.compile] Canonicalize functionalized split slices for fusion pa… ( #57299 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
2026-09-28 20:21:59 -04:00
31cc226401
[ROCm][CI] Run the MLA attention+quant fusion test on ROCm ( #58717 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 14:45:32 -05:00
90a9515006
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands ( #53585 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 12:48:14 -07:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
Andreas Karatzas and OpenAI Codex
26879f3260
[ROCm][Compile] Fuse AITER static FP8 attention output ( #58099 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 03:16:52 -05:00
Andreas Karatzas
0eb42dbbfc
[ROCm][Compile] Support BF16 AsyncTP fusion ( #58098 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 20:36:38 -05:00
Mikko Tukiainen and Cursor
68363d64ec
[ROCm][Perf] Avoid extra reshape kernel in Qwen GDN output norm ( #47842 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-22 14:38:16 +08:00
01f1f58f10
[ROCm][CI] Query HIP device memory for test GPU teardown waits. ( #57450 )
...
Signed-off-by: Sheral Kumar <shekumar@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-20 15:20:39 -05:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
03dc26e639
[Perf][Kernel][Quantization] Fuse ReLU2 with static FP8 activation quantization ( #53793 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Signed-off-by: snordmann <snordmann@nvidia.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 10:26:15 -07:00
78e84261ab
[Performance] Add Triton kernel for Gemma3n sparse GELU ( #48498 )
...
Signed-off-by: BWAAEEEK <jooho414@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-14 07:42:15 +00:00
Qidong Su and OpenAI Codex
9e137d8d23
[Kernel] Enable optimized FlashInfer add-RMSNorm NVFP4 fusion ( #51925 )
...
Signed-off-by: Qidong Su <qidongs@nvidia.com >
Signed-off-by: Qidong Su <soodoshll@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 17:20:32 -07:00
sychen52 and Cyrus Leung
1b2c591cd0
[Bugfix] speedup nvfp4 kv for FMHA ( #55031 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-08 17:25:42 +00:00
8ebc5b0a18
[Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction ( #55643 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: jiahao <jxia77@terpmail.umd.edu >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-08 08:26:07 -07:00
b7624069ff
[Fusion] Manual ActivationQuantFusionPass initial application ( #51415 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: ElizaWszola <ewszola@redhat.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
2026-09-03 13:37:31 -04:00
Khushali Desai
23ab0cfdbc
speculative decoding under tensor parallelism (TP>1) , workspace creation select max hidden dim of target and draft model ( #52193 )
...
Signed-off-by: khushali9 <khushali.desai9@gmail.com >
2026-08-24 21:33:36 +00:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
fbb4c04db4
[ROCm][CI] Speed up test_rocm_aiter_qk_norm_rope_kvcache_fusion ( #53004 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-19 21:20:21 -05:00
Misha Goin and OpenAI Codex
bca7bea240
Remove VLLM_TEST_FORCE_FP8_MARLIN to replace with linear_backend/moe_backend ( #52182 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 12:58:04 -04:00
Stellar鱼
3fa89225bc
[CI/Build] Use file rendezvous for local distributed tests ( #51652 )
...
Signed-off-by: yu-xin-c <2182712990@qq.com >
2026-08-12 10:48:12 +08:00
Jack Hu and Andreas Karatzas
34c1cd20a5
[Bugfix][ROCm] Fix ROCM_AITER_FA & ROCM_AITER_UNIFIED_ATTN QK-Norm+RoPE+KVCache fusion for the packed KV-cache [BLOCKS, HEADS, BLOCK_SIZE, 2*HEAD_DIM] layout ( #49373 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-07 09:54:55 -07:00
Andreas Karatzas
7e724dca89
[ROCm] Fix AITER all-reduce fusion coverage ( #50802 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-06 14:52:23 +08:00
Michael Goin
9cd9f8c20b
Prune redundant tests points in correctness_e2e/[test_sequence_parallel,test_async_tp] ( #51068 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-04 19:18:50 -07:00
Richard Zou and Codex
c27b080d26
[compile] Fix fake kernel return dtype ( #50444 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-07-30 19:50:02 +00:00
BadrBasowid
59e831c09a
[Compilation]Fuse Transformers Residual Add + RMSNorm ( #48757 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-07-30 13:46:36 +00:00
afriedri
f472ab0a4c
Remove triton per group quant [ROCm] [Bugfix] ( #49621 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
2026-07-28 05:53:46 +00:00
Divakar Verma
aaaeda98dc
[CI] fix compile test | refactor VLLM_DISABLE_COMPILE_CACHE for tests ( #49770 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-07-24 23:15:19 -05:00
Jack Hu and Douglas Lehr
2cab53ddee
[Model][Hardware][AMD]: Part 1/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_AITER_FA, and ROCM_AITER_UNIFIED_ATTN ( #42749 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-07-16 17:39:04 -05:00
aoshen02 and OpenAI Codex
3b39fd284a
[Bugfix][Spec Decode] Support heterogeneous QK fusion geometry ( #48671 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-14 22:37:10 -07:00
Hugo Centeno
5f8e73cb8b
[Bugfix] Guard mixed-dtype allreduce RMSNorm quant fusions ( #48330 )
...
Signed-off-by: hcenteno <hugo.centeno@estudiantat.upc.edu >
2026-07-12 09:39:27 +00:00
51878e5b6e
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends ( #44455 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-07-11 11:11:16 -04:00
cbb5f045be
[ROCm][CI] Refresh ROCm base images when docker rocm_base changes ( #46904 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Codex <codex@example.invalid >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Codex <codex@example.invalid >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-07-07 03:10:50 -07:00
jesco and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
598d51153a
[Bugfix][Distributed] Delegate MNNVL allreduce one-shot selection ( #47589 )
...
Signed-off-by: jesco-absolut <team@srswti.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 07:47:06 -07:00
BadrBasowid
740f379fae
[ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator ( #46065 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-07-06 12:16:32 +00:00
xaguilar-amd
09663abde0
[ROCm][MLA] Fuse MLA q/kv RMSNorm + FP8 per-token quant in the FP8 attention path ( #44977 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Signed-off-by: Xavier Aguilar <Xavier.AguilarFruto@amd.com >
2026-07-02 13:00:41 +08:00
Yejing Lai and Kunshang Ji
f651a8a9a4
[XPU][UT]Enable ut qk_norm_rope_fusion ( #42486 )
...
Signed-off-by: Lai, Yejing <yejing.lai@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-01 07:38:03 +00:00
Micah Williamson and Rohan138
d980a3cc6e
[ROCm] Fix AITER_UNIFIED_ATTN Dispatching After AITER Bump ( #46780 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
2026-06-26 02:09:56 -07:00
Charlie Fu and Andreas Karatzas
13b83d77ad
[ROCm][CI] skip test_double_aiter_rms_quant_fusion ( #45967 )
...
Signed-off-by: charlifu <charlifu@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-21 16:53:11 -05:00
Charlie Fu and Rohan Potdar
d955745d58
[ROCm][CI] fix test_rope_kvcache_fusion.py ( #44678 )
...
Signed-off-by: charlifu <charlifu@amd.com >
Co-authored-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com >
2026-06-09 21:53:46 +00:00
Markus Hartikainen and Frida Andersson
80e2c4462d
[ROCm][Compile] Fuse AR + RMSNorm + per-group FP8 quant (+ DSv3.2 indexer fan-out) ( #42864 )
...
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Frida Andersson <fanderss@amd.com >
2026-06-09 12:06:56 +00:00
Mike G and Michael Gschwind
2f27c9a150
Preserve layout-changing clones ( #44574 )
...
Signed-off-by: Michael Gschwind <mgschwind@nvidia.com >
Co-authored-by: Michael Gschwind <mgschwind@nvidia.com >
2026-06-05 20:45:24 -04:00