Anish Shanbhag and OpenAI Codex
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape ( #55838 )
...
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 15:05:28 -04:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving ( #60468 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
2026-10-13 20:17:48 +08:00
JackRao123 and Claude Opus 5.5
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs ( #59709 )
...
Signed-off-by: Jack Rao <jack.rao@baseten.co >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:15:11 +00:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) ( #60602 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 15:30:19 +08:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Nick Riasanovsky
a686c8e1b7
[Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat ( #60502 )
...
Signed-off-by: Nick Riasanovsky <njriasan@meta.com >
2026-10-12 11:38:43 -04:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Warren Deng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fc39317392
[Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys ( #60446 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 14:25:41 +00:00
Mikko Tukiainen and Cursor
81c993658e
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch ( #59732 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-09 21:14:49 +08:00
Chen Cheng and Isotr0py
d095e61c0b
[Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config ( #60763 )
...
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-09 11:35:48 +00:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
Rohan Potdar and Claude Opus 5.5
88a419649e
[ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization ( #60717 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:45:13 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
Shijin Zhang
2274e86842
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash ( #56983 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-10-08 14:41:10 -04:00
1664c4f444
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices ( #54597 )
...
Signed-off-by: Kyle Liang <kylliang@nvidia.com >
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:27:57 +00:00
Elvir Crnčević and Misha Goin
8674a794c7
[Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 ( #56875 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 16:42:15 +00:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
Mikko Tukiainen and Cursor
e2414ec1db
[ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend ( #60021 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-08 15:47:56 +00:00
kliuae
d80ddaac73
[ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj ( #59591 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-08 14:55:20 +00:00
458ba2edf8
[ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization ( #59069 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
2026-10-08 12:01:47 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Nicolò Lucchesi
70bdce6133
[Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer ( #60032 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-10-08 11:05:52 +02:00
fb2ac82458
[ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel ( #56638 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 14:15:18 +08:00
81198e97ba
[Refactor] Abstract DeepGemm MegaMOE backend ( #58053 )
...
Signed-off-by: Hui Shi <kalinshi@tencent.com >
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn >
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu >
2026-10-08 05:05:39 +00:00
1fbbfa1657
[Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec ( #59733 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-10-08 00:33:32 -04:00
Isotr0py and Shanshan Shen
8352b2427f
[Multimodal] Optimize encoder cudagraph padding logic ( #58925 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-08 03:38:09 +00:00
8faac3dd2a
[Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 ( #59655 )
...
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com >
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-10-08 00:10:32 +00:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
e572e02d21
[UX] Skip checkpoint shards an MTP head does not load ( #60152 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:16:19 -04:00
