31 Commits
Author SHA1 Message Date
e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800)
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
2026-10-15 14:51:57 -07:00
Shanshan Shen 2521446ac8 [ROCm][Perf] Enqueue the default chain first in CSA/HCA at small batch sizes for DeepSeek-V4 (#61378)
Signed-off-by: shen-shanshan <467638484@qq.com>
2026-10-14 10:57:26 +08:00
Zheng GongandCursor fcd317cf33 [ROCm][DSv4][Perf] Use the shared prefill chunk plan in the ROCm sparse prefill (#58405)
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-30 15:26:57 +08:00
Mehmet Cagri 70ae0b7435 [ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter (#53492)
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com>
2026-09-29 09:08:18 +08:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-28 05:46:28 -07:00
Fangzhou Ai f9dce295c9 [ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
2026-09-23 01:23:55 +00:00
Tuukka Sarviandlarryli2-amd e34685dfc0 [ROCm][DSv4] Enable DSpark adaptive verification (#52362)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: larryli2-amd <larryli2@amd.com>
2026-09-21 10:03:41 -05:00
yinfengLiuandShanshan Shen 0e110f696d [ROCm][DSV4][Perf] Use FP8 WO_A output projection (#54894)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-20 09:13:55 +00:00
1dc2d854c1 [Feat][Model] Support encoder-side SWA-bounded replay for DeepSeek-V4.1-Flash (#56227)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
2026-09-18 15:01:24 -07:00
Shanshan Shen fdcd42350b [ROCm][Perf] Enable HCA dual-stream overlap for DeepSeek-V4 (#56853)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-17 06:46:00 +00:00
Shanshan Shen bc0f47cd03 [ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 (#57132)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-16 18:58:23 +00:00
Shanshan Shen a6c5d6d0fc [ROCm][Perf] Enable CSA multi-stream overlap for DeepSeek-V4 (#51794)
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-09-14 11:21:30 +00:00
Andreas KaratzasandOpenAI Codex 9dd969da09 [Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-11 09:17:30 -07:00
Matt b4da4d17ae [ROCm][Bugfix] Fix AITER preshuffled FP8 block-scale kernel (#56433)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
2026-09-11 11:44:57 +00:00
385ba6b5b0 [Bugfix][Quantization] Register Quark per-block FP8 scales as weight_scale (#54770)
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-05 10:44:24 +08:00
Shanshan Shen 32ad1400d7 [ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2026-08-27 09:37:54 -07:00
Fangzhou Ai 657f9b9ce2 [ROCm][DSV4][Perf] Fuse DeepSeek V4 C4 compressor GEMMs (#53838)
Signed-off-by: fai <fangzhouai@gmail.com>
2026-08-26 23:36:14 +08:00
ef43e3101b [ROCm][DSV4][Perf] Optimize Triton sparse-MLA decode on gfx950 (#52212)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cursor Grok 4.6 <cursoragent@cursor.com>
2026-08-16 12:16:46 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-08-12 12:27:48 -07:00
Yizhou a311916a29 [MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849)
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
2026-08-11 15:06:22 +08:00
12a34a6bc7 [ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-07-30 13:59:52 -05:00
larryli2-amdandAndreas Karatzas c227aaa3f8 [ROCm] Enable DeepSeek-V4 DSpark speculative decoding on AMD (MI350X / MI355X, gfx950) (#47419)
Signed-off-by: larryli2-amd <larryli2@amd.com>
Signed-off-by: larryli2-amd <Larry.Li@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-07-10 23:22:19 +08:00
Wentao Ye e257faf87d [Refactor] Remove unused rocm kernel combine_topk_swa_indices_ragged (#48158)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-10 09:33:07 -04:00
afdcbd5d39 [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
2026-06-18 12:21:14 +00:00
Woosuk Kwon 2a983c79ac [DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 (#44699)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-06 20:37:56 -04:00
Woosuk Kwon 4efd6ffde0 [DSV4] Refactor DeepseekV4Attention (#44569)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-04 20:23:07 -07:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00
Yongye Zhu 6ab6ffb428 [Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162) 2026-05-26 05:12:54 -10:00
TJian 1806d1adfc [ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (#43385)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
2026-05-24 18:43:08 +08:00
Yongye Zhu 843715739b [Refactor] Extract DeepSeek V4 sparse MLA impl into model folder (#43149) 2026-05-22 10:06:31 -07:00