Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
Miłosz Grunwald and Kunshang Ji
be0823357f
[TEST][XPU][CI] Fix ci uva kernel test instability 2 ( #60404 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 12:09:13 +08:00
Giulio De Pasquale
f1a3c60558
[Feature][Attention] Cover a whole spec-verify in one DiffKV split-KV program ( #59085 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-10-10 11:55:45 +08:00
2d69083c4a
[ROCm][MXFP4] Align the AITER W4A4 MoE intermediate size to 128 ( #55104 )
...
Signed-off-by: xiaofei.zheng <xiaofei.zheng@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-10 11:14:01 +08:00
9f1c783886
[Test] Allow one fp8 step on K bytes in the kpool decode fuzz test ( #60725 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-10 00:26:53 +00:00
Rohan Potdar and Claude Opus 5.5
fa94ad4143
[Test] Allow one bf16 ulp in the fused mHC post/pre residual check ( #60724 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 17:54:06 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
19c0bed584
[Fusion] Support 3D activation quantization fusion and enable it for Pixtral ( #59612 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-09 15:12:03 +08:00
Misha Goin and Claude
e3a29aa9b3
[Bugfix][MoE] Size the permute scratch from its inputs ( #60447 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 15:32:07 -04:00
SeongJun Lee and Misha Goin
fff10d2072
[Attention] Allow an NVFP4 KV cache on SM90 with fa2 ( #60620 )
...
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:16:22 -07:00
Kevin Li and Misha Goin
fc494822ea
[Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 ( #58878 )
...
Signed-off-by: Kevin Li <kevinbean99@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 19:14:35 +00:00
Divakar Verma and Andreas Karatzas
18751bc6a7
[ROCm][CI] Add coverage for AiterExperts token padding: inf/nan garbage rows ( #59334 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-08 19:11:45 +00:00
Misha Goin and Claude
30c7405890
[MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback ( #60079 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-08 11:39:09 -07:00
1664c4f444
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices ( #54597 )
...
Signed-off-by: Kyle Liang <kylliang@nvidia.com >
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:27:57 +00:00
stefankoncarevic
f4dde3132c
[CI][ROCm] Reuse AITER's prebuilt MoE module in the modular-kernel harness ( #60633 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-10-08 11:18:05 -05:00
JartX
994b8b77be
[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout ( #60388 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-10-08 15:06:39 +00:00
Andy Lo
ba77c4c130
[Bugfix][Attention] Preserve small FP8 softmax weights in Triton attention ( #60156 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-10-08 12:35:22 +02:00
Rehan Khan and Bob
fd0216d3b5
[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias ( #60221 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Bob <bob@ibm.com >
2026-10-08 15:15:53 +08:00
fb2ac82458
[ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel ( #56638 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 14:15:18 +08:00
Jiahan Chang (Cyrus)
816173ca9d
[Kernel] Remove WAR for LL CuTeDSL BF16 GEMM ( #60434 )
...
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
2026-10-08 05:38:07 +00:00
Simon Danielsson and Douglas Lehr
87d9996abe
[ROCm][Perf][GLM-5.3-Flash] BF16 splitk kernel for sparse MLA decode ( #58584 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-07 21:37:51 -05:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
554340f3d3
[Attention] Support NVFP4 KV cache on SM8x and SM12x with FlashInfer ( #46963 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 13:24:21 -04:00
00CC
5281e49908
[ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel ( #52619 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-10-07 13:21:43 +00:00
8a26869bca
[Bugfix] Index Mamba align state copies by request slot in MRV2 ( #60210 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com >
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com >
2026-10-07 02:32:46 +00:00
956960f85f
[Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies ( #60215 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 01:51:12 +00:00
Robert Shaw and Claude Code
283d76d56a
[Bugfix] Widen ep_gather output offset to int64 ( #58373 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-07 01:45:04 +00:00
fe51194e2d
[Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse ( #60194 )
...
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 01:35:12 +00:00
5ab4e4445b
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts ( #59128 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
d1f3d8b870
[Quantization] Prefer Humming before Marlin backends on SM90 ( #56997 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:31:35 -07:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
Mia Ojeda
3313825983
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step ( #59992 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
2026-10-06 19:25:51 +08:00
roikoren755 and zjy0516
e11962fc1c
[Feat][Mamba2] Enable internal prefill checkpoints ( #57329 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-06 07:53:51 +00:00
edfccc9469
[ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 ( #59132 )
...
Signed-off-by: root <jeffaw99@hotmail.com >
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com >
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com >
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 07:53:47 +00:00
40c86ae1ce
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x ( #58877 )
...
Signed-off-by: lazymio <mio@lazym.io >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 01:49:13 -04:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
21d93d0d8c
[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash ( #58128 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-05 21:18:29 -07:00
4e198cef83
[TEST][XPU][CI] Fix ci uva kernel test instability ( #60041 )
...
Signed-off-by: Your Name <you@example.com >
Co-authored-by: Your Name <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-06 09:19:01 +08:00
97bd8c74eb
[ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache ( #56720 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 17:57:46 -07:00
Mason and Yongye Zhu
6dd813e0b7
[Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend ( #59571 )
...
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-10-05 16:24:14 -07:00
akii96
182247257f
[ROCm][Perf] Use a stacked hipBLASLt GEMM for the bf16x3 router ( #52668 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
2026-10-05 17:28:07 +00:00
roikoren755 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
51eeb0c58f
[Bugfix] Load stacked expert weights for non-gated MoE ( #59031 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 15:26:55 +03:00
042ab0305c
[Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections ( #59533 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com >
2026-10-05 07:03:29 +00:00
e8a53a55e6
[Bugfix][Attention] Support int4_per_token_head on non-power-of-two head sizes ( #56198 )
...
Signed-off-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com >
Co-authored-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-05 04:52:39 +00:00
49e0f47978
[Bugfix][DSv4.1] Keep the compressor ring out of the null block ( #58560 )
...
Signed-off-by: bluemelov <hannes.b2411@eva.space >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: bluemelov <hannes.b2411@eva.space >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 01:49:41 +00:00
00CC
f98d1fc493
[ROCm][Bugfix] Fix ROCM_ATTN sliding-window boundary ( #59550 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-10-04 13:48:46 +00:00
4ac0d0eac2
[ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism ( #54706 )
...
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com >
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
2026-10-03 23:44:17 +08:00
Divakar Verma
faa9860dbd
[ROCm][CI][AiterExperts] Add test coverage for hidden/intermediate padding correctness ( #59333 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-10-03 01:53:37 +00:00
