221 Commits
Author SHA1 Message Date
e80b6c642f [ROCm][DSv4] Overlap the unfused shared expert with AITER MegaMoE (#61819)
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
2026-10-18 22:39:38 +08:00
026f6ab268 [Model] DeepSeek-V4/V4.1 VL: stream language-model weights instead of sorting the checkpoint (#58857)
Signed-off-by: git bisector <gitbisector@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-17 09:37:56 +00:00
Micah WilliamsonandCursor a0e7137c4b [CI][ROCm][Bugfix] DeepSeek V4.1: don't pass expert intermediate padding to AITER (#61960)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-17 04:25:39 +00:00
e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
2026-10-15 14:51:57 -07:00
f8460ef236 [Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371)
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
2026-10-15 13:58:25 -07:00
Rohan PotdarandClaude Opus 5.5 105605b8e9 [ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282)
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:10:12 -05:00
4d58fa254a [MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 19:02:01 +00:00
Micah Williamson e8e1e96196 [ROCm][DSv4] AITER MegaMoE Fixed-Slot Dispatch For DeepSeek-V4 (#61262)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
2026-10-15 19:10:59 +08:00
jimmy-adams 365aaa77bd [Bugfix][DSv4] Remap Quark MTP layer_quant_config onto draft prefixes (#56054)
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
2026-10-15 18:51:57 +08:00
Harry MellorClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kevin H. Luu
b965ee564d [Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 01:08:08 -07:00
Shanshan Shen f46021a7fa [ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
2026-10-15 11:16:48 +08:00
984c242ede [Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-14 13:33:35 -07:00
Ronaldandaoshen02 19aba3e4d5 [Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-10-14 10:17:58 +00:00
Shanshan Shen 2521446ac8 [ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 (#61378)
Signed-off-by: shen-shanshan <467638484@qq.com>
2026-10-14 10:57:26 +08:00
Wenbin ChenandMisha Goin 6463d04601 Fix write-after-write hazard in xpu KV RoPE triton kernel (#46685)
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-10-13 22:17:14 +00:00
c5bda658d2 [Model][MoE] Load MegaMoE experts directly into the DeepGEMM layout (#60096)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Aaron Hao <hao-aaron@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aaron Hao <hao-aaron@users.noreply.github.com>
2026-10-13 11:52:08 -07:00
JackRao123andClaude Opus 5.5 877f6df889 [Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs (#59709)
Signed-off-by: Jack Rao <jack.rao@baseten.co>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:15:11 +00:00
Yongye ZhuandClaude Opus 5.5 a6009ea85b [Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 23:24:43 -07:00
Ilya Markov a281ad32f0 [EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
2026-10-12 10:03:09 -04:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004)
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-10-12 15:53:16 +08:00
Warren Deng 59ffd73e85 [Bugfix][Kernel] Restore FP fusion for indexer Q quantization (#60716)
Signed-off-by: Warren Deng <warrdeng@meta.com>
2026-10-09 09:46:41 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
81198e97ba [Refactor] Abstract DeepGemm MegaMOE backend (#58053)
Signed-off-by: Hui Shi <kalinshi@tencent.com>
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn>
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu>
2026-10-08 05:05:39 +00:00
f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740)
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-10-07 22:01:09 +00:00
e572e02d21 [UX] Skip checkpoint shards an MTP head does not load (#60152)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:16:19 -04:00
Micah WilliamsonandFangzhou Ai f6e3944e31 [ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
2026-10-07 09:35:53 -05:00
aoshen02andClaude Opus 5.5 4a30c4cad0 [Model][DeepSeek-V4] Make MegaMoE shared-expert finalize independent of linear post-load order (#59927)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 08:39:49 +00:00
Yongqi WangandKunshang Ji 155d23cb00 [Bugfix][XPU] Make DeepSeek V4 FP8 sparse decode graph-capturable (#59159)
Signed-off-by: Yongqi Wang <yongqi.wang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-10-04 18:31:15 +08:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049)
Signed-off-by: jdebache <jdebache@nvidia.com>
2026-10-02 15:43:33 -04:00
Yida WengandGrok 17e9295dd5 [BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size (#59271) (#59373)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Co-authored-by: Grok <grok@x.ai>
2026-09-30 15:09:11 +00:00
Varun Sundar Rabindranath 5463fe4962 [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) 2026-09-30 09:11:03 -04:00
Zheng GongandCursor fcd317cf33 [ROCm][DSv4][Perf] Use the shared prefill chunk plan in the ROCm sparse prefill (#58405)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-30 15:26:57 +08:00
Mehmet Cagri 70ae0b7435 [ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter (#53492)
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com>
2026-09-29 09:08:18 +08:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-28 05:46:28 -07:00
BaoYunkai f6877f563c [Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter (#57568)
Signed-off-by: BaoYunkai <ybao@amd.com>
2026-09-28 20:16:38 +08:00
Kevin H. Luu 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) 2026-09-26 11:37:36 -04:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071)
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-25 20:51:18 -05:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 23:26:50 -07:00
44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-23 22:45:23 -07:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
2026-09-23 20:29:06 -07:00
Fangzhou Ai f9dce295c9 [ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
2026-09-23 01:23:55 +00:00
f92b78f6ef [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
2026-09-22 01:07:17 -07:00
Shanshan Shen 42a85a4976 [ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-22 04:45:53 +00:00
Tuukka Sarviandlarryli2-amd e34685dfc0 [ROCm][DSv4] Enable DSpark adaptive verification (#52362)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: larryli2-amd <larryli2@amd.com>
2026-09-21 10:03:41 -05:00
Woosuk KwonandCodex 9a70c233cd [Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 20:28:06 -07:00
Isotr0py 27757dde02 [DSV4.1] Add encoder cuda graph support for deepseek-v4.1-flash (#56625)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-20 21:51:13 +08:00
yinfengLiuandShanshan Shen 0e110f696d [ROCm][DSV4][Perf] Use FP8 WO_A output projection (#54894)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-20 09:13:55 +00:00
1dc2d854c1 [Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
2026-09-18 15:01:24 -07:00
Woosuk KwonandCodex 50d812b66a [Perf][DSV4.1] Optimize MegaMoE staging and NVFP4 cache gathers (#57604)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
2026-09-18 12:57:19 -07:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:29:11 +01:00