Andreas Karatzas and Claude
e96c822660
[Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer ( #60990 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:13:48 -05:00
lucas-santos-amd and Douglas Lehr
e534024e49
[ROCm][Bugfix][MLA] Fix FP8 MLA prefill unpad for interleave-padded heads ( #60907 )
...
Signed-off-by: Lucas Santos <Lucas.Santos@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-14 14:34:28 -04:00
Kevin H. Luu and OpenAI Codex
0b6d07cbe2
[CI/Build] Use FileStore for DeepEP test rendezvous ( #56595 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 14:29:12 -04:00
Zheng Gong and Cursor
355553ac22
[ROCm][Perf] Drop two dead per-step ops from the AITER sparse MLA metadata build ( #58354 )
...
Signed-off-by: ZhengGong-amd <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-14 11:15:51 -05:00
Ashish Rao and Vadim Gimpelson
b4b7eca0ef
[Bugfix][Quantization] Clamp FP8 scales before UE8M0 rounding ( #58032 )
...
Signed-off-by: Ashish Rao <asrao@nvidia.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-10-14 13:56:42 +00:00
xuebwang
81503550f3
[ROCm] Disable AITER FP8/FP4 BMM for MLA by default ( #53123 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-10-14 10:46:02 +00:00
f9b65c30c7
[ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 ( #60645 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 10:11:57 +00:00
Jiangyun Zhu and Claude Opus 5.5
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend ( #60050 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 14:49:46 +08:00
2308b6f00b
[Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE ( #58897 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-14 10:48:05 +08:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ad8d39cbab
[Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k ( #59166 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-14 07:49:52 +08:00
Anirudh Ravichandran
e90158d912
[Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE ( #60733 )
2026-10-13 17:02:17 -04:00
djramic
93d0bd9d50
[ROCm][CI] Drop the -inf padding check from the AITER MQA logits test ( #61420 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-13 15:49:30 -04:00
Anish Shanbhag and OpenAI Codex
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape ( #55838 )
...
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 15:05:28 -04:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
Roberto L. Castro and Claude Opus 5.5
7fad7db330
[Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) ( #44343 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 12:38:46 -04:00
6849ca696a
[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x ( #59604 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Luke Alonso <lalonso@gmail.com >
2026-10-13 09:06:33 -07:00
Elvir Crnčević
b9b401b2ef
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch ( #53911 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-10-13 11:10:38 -04:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Vadim Gimpelson and Claude
cd7517fb17
[Bugfix] Fix ReplaySSM ring shape assertion when TP does not divide Mamba2 n_groups ( #60704 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 11:31:53 +00:00
JackRao123 and Claude Opus 5.5
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs ( #59709 )
...
Signed-off-by: Jack Rao <jack.rao@baseten.co >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:15:11 +00:00
c7a305ec10
[Bugfix][Mamba] Store conv_kernel - 1 inputs in Mamba2 prefill checkpoints ( #60797 )
...
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com >
2026-10-13 10:09:17 +00:00
Thien Tran and OpenAI Codex
656d0d4506
[Perf] Tune Triton GDN decode tiles on SM100-family GPUs ( #61355 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-13 17:01:12 +08:00
Vadim Gimpelson and Claude
183625d4f9
[Bugfix] Fall back to auto for unquantized MoE layers when --moe-backend is marlin ( #60219 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 08:48:13 +00:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
0d2689e0df
[ROCm][Perf] Skip redundant sparse index remap on non-indexer layers ( #51309 )
...
Signed-off-by: Sriram Kumar <sriramkumar.kishorekumar@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 05:25:56 +00:00
Divakar Verma and Claude Sonnet 5
f7ba007b5e
[ROCm][MoE] Split AiterExperts by weight-quantization key ( #60322 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-13 00:13:33 -05:00
c6566bb9f6
[ROCm][MLA] Keep DCP prefill context in FP8 through AllGather ( #59070 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com >
2026-10-13 09:20:38 +08:00
98f7211162
[Attention][Spec Decode] Multi-query Triton split-K for spec-decode verify, with empty-segment/padding hardening ( #56148 )
...
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com >
Signed-off-by: Venky <23023424+venkywonka@users.noreply.github.com >
Co-authored-by: OpenAI GPT-6 Astra <noreply@openai.com >
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 04:37:56 +04:00
189bbfb9f4
Add MoNe experts pruning support ( #50415 )
...
Signed-off-by: Ilya Boytsov <ieboytsov@nebius.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Ilya Boytsov <ieboytsov@nebius.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 23:18:43 +00:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
e0317b28ce
[Feature] Support batch invariance on ROCm ( #52231 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-12 10:53:26 -05:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Shang Wang and Claude Opus 5.5
716961cf1f
[Bugfix] Drop enable_in_kernel_fc2_reduce from FlashInfer CuteDSL MegaMoE config ( #59900 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 09:05:09 -05:00
Ilya Markov
b3ee560ae8
[Bugfix][MoE] Apply expert_map remapping for CUTLASS FP4 MOE expert parallel ( #59042 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:02:56 -04:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Miłosz Grunwald and Kunshang Ji
be0823357f
[TEST][XPU][CI] Fix ci uva kernel test instability 2 ( #60404 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 12:09:13 +08:00
Giulio De Pasquale
f1a3c60558
[Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program ( #59085 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-10-10 11:55:45 +08:00
2d69083c4a
[ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 ( #55104 )
...
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-10 11:14:01 +08:00
9f1c783886
[Test] Allow one fp8 step on K bytes in the kpool decode fuzz test ( #60725 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-10 00:26:53 +00:00
Rohan Potdar and Claude Opus 5.5
fa94ad4143
[Test] Allow one bf16 ulp in the fused mHC post/pre residual check ( #60724 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 17:54:06 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
19c0bed584
[Fusion] Support 3D activation quantization fusion and enable it for Pixtral ( #59612 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 15:12:03 +08:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
SeongJun Lee and Misha Goin
fff10d2072
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 ( #60620 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:16:22 -07:00
Kevin Li and Misha Goin
fc494822ea
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 ( #58878 )
...
Signed-off-by: Kevin Li <kevinbean99@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 19:14:35 +00:00
Divakar Verma and Andreas Karatzas
18751bc6a7
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows ( #59334 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 19:11:45 +00:00
