Rohan Potdar and Claude Opus 5.5
9a648b4649
[ROCm][Kimi-K3] Don't read W_UK_T before weight loading in the MLA wrapper ( #61654 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 21:44:55 +00:00
984c242ede
[Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod ( #59815 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:33:35 -07:00
Wei Zhao and Jee Jee Li
4975fda882
[Dependency] Upgrade FlashInfer to 0.7.1 ( #60948 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-10-14 12:31:08 -07:00
Andreas Karatzas and Claude
e96c822660
[Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer ( #60990 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-14 14:13:48 -05:00
Simon Danielsson
3f729dafcc
[Perf][ROCm][GLM-5.3-Flash] AITER prefill indexer topk ( #60753 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-10-14 13:48:56 -05:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
77154ca611
[Perf][Model][Qwen4Exp] QSA: decode four FP8 e4m3 bytes per register below sm_89 ( #60992 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-14 13:06:16 +08:00
Shanshan Shen
2521446ac8
[ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 ( #61378 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-14 10:57:26 +08:00
Stefano Castagnetta and Kartikey Vyas
4e0681ed56
[Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings ( #56273 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai >
2026-10-14 08:24:55 +08:00
Tim002 and Thien Tran
96fc2da315
[Perf][Qwen4Exp] Add SM121 TP=2 merged QSA projection plans ( #61437 )
...
Signed-off-by: TimRen01 <HolaTim@hotmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 22:58:44 +00:00
Wenbin Chen and Misha Goin
6463d04601
Fix write-after-write hazard in xpu KV RoPE triton kernel ( #46685 )
...
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-13 22:17:14 +00:00
Andreas Karatzas and Claude
67516b6c42
[Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading ( #61207 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-13 14:55:57 -05:00
Anish Shanbhag and OpenAI Codex
3f550129e8
[Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape ( #55838 )
...
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-13 15:05:28 -04:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
253e02b95e
[Perf] Warm up DeepGEMM MegaMoE kernels before serving ( #60468 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
2026-10-13 20:17:48 +08:00
JackRao123 and Claude Opus 5.5
877f6df889
[Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs ( #59709 )
...
Signed-off-by: Jack Rao <jack.rao@baseten.co >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:15:11 +00:00
bc3ddb198a
[Model][Qwen4Exp] FP8 QSA main KV cache on Ampere (sm_80, sm_86) ( #60602 )
...
Signed-off-by: kimseunghyun <hellonypcteam@gmail.com >
Signed-off-by: kimseunghyun <incongnito12@gmail.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: kimseunghyun <hellonypcteam@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-13 15:30:19 +08:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
Misha Goin and Claude
d79b52b182
[Misc] Consolidate csrc shared headers and prune dead code ( #60721 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 09:46:22 -07:00
TJian
50b21a2667
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA ( #49263 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-10-12 15:50:28 +00:00
Nick Riasanovsky
a686c8e1b7
[Triton] Migrate Triton kernels off tl.make_block_ptr for triton-main compat ( #60502 )
...
Signed-off-by: Nick Riasanovsky <njriasan@meta.com >
2026-10-12 11:38:43 -04:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Ilya Markov
a281ad32f0
[EPLB] Add EPLB support to dspark draft models (DeepSeekV4) ( #57549 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-10-12 10:03:09 -04:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c75cb63cd2
[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next ( #56322 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-12 20:56:44 +08:00
AMD-yanfeiwang and aoshen02
e3bbaf2369
[ROCm][DSv4] Fuse the shared expert into AITER MegaMoE ( #61004 )
...
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-12 15:53:16 +08:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
f26d9f432c
[Bugfix][GLM-5.3-Flash] Honor --mamba-ssm-cache-dtype for the KDA recurrent state ( #59147 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-11 17:55:00 +00:00
3e8e1364ff
[Perf][GLM-5.3-Flash] Shard long-context indexer prefill rows across TP ( #54951 )
...
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com >
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: Codex <noreply@openai.com >
Co-authored-by: root <root@rjob-fb29567b88fec0a9-88c9a4c797eb2ec1-0.caizheng.ailab-sys.svc.pjlab.local >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-11 10:43:37 -04:00
Yongye Zhu and Claude Opus 5.5
2eb491e18b
[Perf][MiniMax-M3] MSA prefill attends with the E4M3 query; bump fmha_sm100 to 968061a ( #60684 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 13:40:45 +08:00
SeongJun Lee and lesj0610
a98247ab4d
[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 ( #59853 )
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
Signed-off-by: lesj0610 <lesj0610@gmail.com >
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-10-10 06:51:52 +00:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Warren Deng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fc39317392
[Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys ( #60446 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 14:25:41 +00:00
Mikko Tukiainen and Cursor
81c993658e
[ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch ( #59732 )
...
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-09 21:14:49 +08:00
Chen Cheng and Isotr0py
d095e61c0b
[Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config ( #60763 )
...
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-09 11:35:48 +00:00
Warren Deng
59ffd73e85
[Bugfix][Kernel] Restore FP fusion for indexer Q quantization ( #60716 )
...
Signed-off-by: Warren Deng <warrdeng@meta.com >
2026-10-09 09:46:41 +00:00
Harry Mellor and Claude Opus 5.5
2c99ee9333
[Chore] Remove vendored code made redundant by Transformers 5.16.1 ( #60623 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 09:29:03 +00:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
82e89e9944
[Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan ( #60776 )
...
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 07:18:50 +00:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
Rohan Potdar and Claude Opus 5.5
88a419649e
[ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization ( #60717 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 04:45:13 +00:00
Yifan Qiao and Claude Opus 5.5
240785b82c
[Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block ( #59528 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 20:44:04 +00:00
Shijin Zhang
2274e86842
[Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash ( #56983 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-10-08 14:41:10 -04:00
1664c4f444
[Bugfix][MiniMax-M3] Guard sparse decode sentinel indices ( #54597 )
...
Signed-off-by: Kyle Liang <kylliang@nvidia.com >
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-08 18:27:57 +00:00
Elvir Crnčević and Misha Goin
8674a794c7
[Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 ( #56875 )
...
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 16:42:15 +00:00
9f3d70e081
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) ( #55686 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 12:04:46 -04:00
