23 Commits
Author SHA1 Message Date
JackRao123andClaude Opus 5.5 877f6df889 [Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs (#59709)
Signed-off-by: Jack Rao <jack.rao@baseten.co>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:15:11 +00:00
Varun Sundar Rabindranath 5463fe4962 [KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652) 2026-09-30 09:11:03 -04:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
2026-09-23 20:29:06 -07:00
Roberto L. CastroandCodex 120ec4ebd2 [5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-12 00:05:59 -04:00
Wentao Ye 5fe77aecfc [Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-11 08:17:02 -07:00
Roberto L. CastroandCodex 6ddbab03de [4/N][warmup][DSv4] Migrate common attention kernels (#50176)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
2026-09-08 10:25:48 -04:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
2026-08-21 17:47:49 -07:00
Woosuk KwonandOpenAI Codex f1178f3a06 Revert DSv4 eager workspace reuse (#52836)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-18 17:02:59 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
Wentao Ye 37e370fe93 [DSv4 Perf] Skip empty c128 kernel launch, around 2x kernel performance improvement. (#48957)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-22 10:55:20 -04:00
kliuae eb33ff34dd [ROCm][Perf] DSv4 two-stage compressor kernel for HCA prefill (#47718)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-07-15 23:31:51 +00:00
Jeff (Junze) MaandLucas Wilkinson 80eb01e93d [Bugfix] DSV4 TP16 garbage output (#47493)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-07-07 21:04:33 -07:00
Wentao Ye f70caef48b [Perf] Cache token_to_req_indices for dsv4, 5x~6x kernel performance improvement (#47474)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-07-06 11:17:46 -04:00
44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-06-22 11:54:14 -07:00
Ma Jian eebce65756 [XPU]feat: add DeepSeek-V4 XPU attention decode path (#42953)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
2026-06-08 13:27:12 +08:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00
Harry Mellor 41688e2dc7 Fix early CUDA init (#43791)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-05-27 09:30:11 -07:00
Woosuk Kwon adaa5e455a [DSv4] Refactor compressor & Fix ROCm compatibility (#43710)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-26 19:56:46 -07:00
Woosuk Kwon 193ce8812e [DSv4] Drop _get_compressed_kv_buffer in DeepseekCompressor (#43690)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-26 10:11:25 -07:00
a37e47100c Add CuTe DSL sparse compressor support (#43584)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-05-26 00:11:12 -07:00
Ma JianandKunshang Ji 8de5cabeb7 [XPU]fix: add XPU platform guards to DeepSeek-V4 ops (#42950)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-23 06:29:45 +08:00
Woosuk Kwon b14be81c1f [Model Refactoring] Move deepseek_v4_ops to models/deepseek_v4 [3/N] (#43073)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-19 00:52:54 -07:00
Woosuk Kwon 87b08c5f64 [Model Refactoring] Move DeepSeek V4 layers to models/deepseek_v4/ [2/N] (#43039)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-18 21:00:58 -07:00