026f6ab268
[Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint ( #58857 )
...
Signed-off-by: git bisector <gitbisector@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-17 09:37:56 +00:00
fa75d3456d
[Feature][PCP] Add DeepSeek-V4.1-Flash support ( #59857 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 23:17:51 -07:00
Shuolei Wang
05ad0a8ace
[Perf][DSv4.1] Split fused_wqa_wkv by token across TP ranks on SM90 ( #58872 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-10-16 02:31:34 +00:00
Ronald and aoshen02
d12d5460a5
[Bugfix][MoE] Keep routing rows during DeepSeek V4.1 decoder replay ( #61643 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-15 14:02:06 +00:00
Ronald and aoshen02
19aba3e4d5
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload ( #57797 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-14 10:17:58 +00:00
c5bda658d2
[Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout ( #60096 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com >
2026-10-13 11:52:08 -07:00
frida-andersson
c077dc663d
[ROCm][Perf][DSv4.1] Enable decoder SWA bounded replay ( #59039 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
2026-10-13 02:19:14 +00:00
Shang Wang and Claude Opus 5.5
ffec221276
[Bugfix][DSv4.1] Match decoder-replay input_ids dtype for FlashInfer MegaMoE ( #61041 )
...
Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 14:23:00 +00:00
Giancarlo Delfin and Rishi Puri
36f3972812
[Spec Decode] Skip metadata build for dflash/dspark when all model backends support it ( #60692 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Rishi Puri <20074092+puririshi98@users.noreply.github.com >
2026-10-12 00:09:10 -07:00
02e55d2af3
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots ( #57169 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-09 15:48:37 -04:00
efaca9547d
[Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 ( #59923 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 15:30:18 +08:00
ad73a4740d
[Perf][DSv4.1] Trim the last KV-source layer in decoder replay ( #59894 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Juntian Liu <Juntianl777@gmail.com >
2026-10-09 06:07:49 +00:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
Juntian Liu and Claude Opus 5.5
4c2d277643
[Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups ( #59327 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-01 11:54:44 +08:00
Yifan Qiao and Claude Fable 5.1
4b2e1cfa7b
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 ( #58132 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 08:25:46 +00:00
Juntian Liu and Claude Opus 5.5
a9eafde59c
[Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 ( #58634 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-26 21:16:12 -07:00
Yongye Zhu and Claude Opus 5.5
77871126f9
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary ( #58586 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 03:08:36 +08:00
6936e77ba4
[Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages ( #56926 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 10:52:02 +08:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00
Jhao-Ting Chen
4fb767ff0a
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection ( #53175 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
2026-09-23 20:29:06 -07:00
a6131b0e65
[Bugfix][Engram] Fall back when /dev/shm is absent before sharing tables ( #57914 )
...
Signed-off-by: Juntian Liu <juntianl@inferact.ai >
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 00:19:09 +08:00
f92b78f6ef
[Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter ( #57428 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-09-22 01:07:17 -07:00
76ffb0d374
[Model][Engram] Share host tables across co-located DP replicas by default ( #57651 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-21 08:26:56 +00:00
Woosuk Kwon and OpenAI Codex
67513c8b67
[Bugfix][DSV4.1] Restrict mHC overlap to full CUDA graphs ( #57874 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-21 01:16:59 -07:00
Woosuk Kwon and Codex
9a70c233cd
[Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation ( #57643 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-20 20:28:06 -07:00
Woosuk Kwon and Codex
9b49f92344
[Perf][DSV4.1] Overlap mHC coefficients for small TP batches ( #57603 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-20 17:33:17 -07:00
Isotr0py
27757dde02
[DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash ( #56625 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-20 21:51:13 +08:00
1dc2d854c1
[Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash ( #56227 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-18 15:01:24 -07:00
Woosuk Kwon and Codex
50d812b66a
[Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers ( #57604 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-18 12:57:19 -07:00
Woosuk Kwon and Codex
80447d2765
[Bugfix][DSv4.1] Fix FlashInfer DSpark non-causal attention ( #57432 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-17 15:14:37 -07:00
41f9104fa6
[DSv4.1] Integrate Mega-Gate from DeepGEMM ( #56266 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-17 12:06:36 -07:00
9f9e1dac26
[Bugfix][Model] Restore causal image SWA for DeepSeek V4.1 ( #57152 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-16 18:09:00 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
d6a1677d55
[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache ( #56935 )
...
Signed-off-by: Yongye Zhu <yongye@inferact.ai >
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Yunxiao Ning <73625538+foraxe@users.noreply.github.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-16 03:07:50 -07:00
6ecd97f1b7
[Perf][DSpark] Add KV-only context insertion across V4.1 cache formats ( #56441 )
...
Signed-off-by: 0z5a <0z5a@users.noreply.github.com >
Signed-off-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: 0z5a <0z5a@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-15 21:34:55 -07:00
bb55077411
[Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255 ) ( #56962 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-15 04:01:06 -07:00
073f883b56
[Perf][DSpark] Collapse DeepSeek-V4.1 draft states before SP all-gather ( #56903 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-15 02:58:57 -07:00
e6eb0d120c
[Model][DSv4.1] Store the whole KV in MXFP8 (FlashMLA V4.1 record) ( #56893 )
...
Signed-off-by: Yongye Zhu <yongye@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-14 20:12:55 -07:00
Yongye Zhu
5372e72a98
[Perf][DSv4.1] Fold the mHC post block into the delayed pre projection ( #56633 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-14 18:57:37 +00:00
Bugen Zhao
cf1584f373
[Refactor] Normalize DeepSeek V4.1 model package naming ( #56741 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-13 23:17:50 -07:00