026f6ab268
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint ( #58857 )
...
Signed-off-by: git bisector <gitbisector@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-17 09:37:56 +00:00
Micah Williamson and Cursor
a0e7137c4b
[CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER ( #61960 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-17 04:25:39 +00:00
fa75d3456d
[Feature][PCP] Add DeepSeek-V4.1-Flash support ( #59857 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 23:17:51 -07:00
Shuolei Wang
05ad0a8ace
[Perf][DSv4.1] Split fused_wqa_wkv by token across TP ranks on SM90 ( #58872 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-10-16 02:31:34 +00:00
f8460ef236
[Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache ( #61371 )
...
Signed-off-by: juntianl <juntianl@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-15 13:58:25 -07:00
Ronald and aoshen02
d12d5460a5
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay ( #61643 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-15 14:02:06 +00:00
0951fc183c
[ROCm][DSv4.1] Disambiguate ROCm attention class names ( #58689 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 21:38:39 +08:00
b965ee564d
[Chore] Use upstream Transformers DeepseekV4Config ( #61392 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 01:08:08 -07:00
Tuukka Sarvi and Kevin H. Luu
10d2c6083d
[ROCm][DSv4.1] Enable DSpark adaptive verification ( #58151 )
...
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 12:05:43 +08:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
Yongye Zhu and Claude Opus 5.5
a6009ea85b
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer ( #61149 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-12 23:24:43 -07:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
Fangzhou Ai and Claude Opus 5.5
cc2a6eea1a
[ROCm][Perf][DSv4.1] Mono decode on every backbone layer ( #61082 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 10:16:50 +08:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
Fangzhou Ai and Claude Opus 5.5
193922d6ff
[ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X ( #60397 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:18 -05:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
Chao Lei
bc713cf8cc
[Bugfix]MooncakeConnect add query timeout and retry ( #55923 )
2026-10-09 09:19:33 +00:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
fb2ac82458
[ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel ( #56638 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-08 14:15:18 +08:00
Isotr0py and Shanshan Shen
8352b2427f
[Multimodal] Optimize encoder cudagraph padding logic ( #58925 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-10-08 03:38:09 +00:00
8faac3dd2a
[Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 ( #59655 )
...
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com >
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-10-08 00:10:32 +00:00
f38c6778ce
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 ( #56740 )
...
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-07 22:01:09 +00:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
97bd8c74eb
[ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache ( #56720 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 17:57:46 -07:00
49e0f47978
[Bugfix][DSv4.1] Keep the compressor ring out of the null block ( #58560 )
...
Signed-off-by: bluemelov <hannes.b2411@eva.space >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: bluemelov <hannes.b2411@eva.space >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 01:49:41 +00:00
Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
840c7c2f4b
[Bugfix][Engram] Fix intermittent Triton 3.8 crash in the lookup kernel ( #59639 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-01 19:38:55 +00:00
Juntian Liu and Claude Opus 5.5
4c2d277643
[Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups ( #59327 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-01 11:54:44 +08:00
Zheng Gong
8ee4069033
[ROCm][DSv4.1][Perf] Use the shared prefill chunk plan in the ROCm sparse prefill ( #58539 )
...
Signed-off-by: Zheng Gong <zgong@amd.com >
2026-09-30 15:31:51 +08:00
ac7f3e11ea
[Bugfix] Profile maximum DeepSeek V4.1 vision features ( #57898 )
...
Signed-off-by: ahmed xijiaat <52128022+xijiaat@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 15:21:27 +00:00
Yifan Qiao and Claude Fable 5.1
4b2e1cfa7b
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 ( #58132 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 08:25:46 +00:00
Juntian Liu and Claude Opus 5.5
30f5c01d4e
[DSv4.1] Avoid runtime recompiles of _ring_slot_mapping_kernel ( #59119 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-29 07:31:30 +00:00
63b9931b73
[ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel ( #58671 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-29 14:37:57 +08:00
Muhammad Ahmed
0af34418e9
[ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel ( #58655 )
...
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com >
2026-09-29 10:19:10 +08:00
Mehmet Cagri
70ae0b7435
[ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter ( #53492 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com >
2026-09-29 09:08:18 +08:00
Shanshan Shen
7ba3df63cb
[ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform ( #58983 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-28 05:46:28 -07:00
Shanshan Shen
44af287ebe
[ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash ( #57407 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-27 23:48:01 +08:00
Juntian Liu and Claude Opus 5.5
a9eafde59c
[Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 ( #58634 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-26 21:16:12 -07:00
Yongye Zhu and Claude Opus 5.5
77871126f9
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary ( #58586 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 03:08:36 +08:00
Kevin H. Luu
7d8c5fe9a9
[Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata ( #58499 )
2026-09-26 11:37:36 -04:00
xiao-llm and Cursor
5840d95284
[Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support ( #57071 )
...
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-25 20:51:18 -05:00
Shuolei Wang
4bb804cc5b
[Perf][DSv4.1] Shard the Engram wkv projection across TP ranks ( #58678 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-09-26 01:15:57 +00:00
6936e77ba4
[Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages ( #56926 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:52:02 +08:00
Juntian Liu
609731e8f5
[Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path ( #57679 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
2026-09-25 01:39:03 +00:00
Fangzhou Ai and Claude Opus 5.5
721d0e5c11
[ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM ( #58456 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 11:58:14 +00:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00