20 Commits
Author SHA1 Message Date
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 23:26:50 -07:00
44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-23 22:45:23 -07:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
2026-09-23 20:29:06 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Isotr0py 1356635d83 [New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-02 11:09:09 -07:00
Ilya Markov 25efcfa788 [Attention] Enable adaptive verification for FLASHINFER_MLA_SPARSE_DSV4 (#52724)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
2026-09-01 13:43:28 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
2026-08-21 17:47:49 -07:00
Woosuk KwonandOpenAI Codex f1178f3a06 Revert DSv4 eager workspace reuse (#52836)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-18 17:02:59 -07:00
Gabriel Wu 97388c44f9 [Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538) 2026-08-15 07:01:32 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-08-12 12:27:48 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> df71917cf1 [DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
Jeff (Junze) MaClaude Opus 4.8mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Yongye Zhu
b49eaf205a [DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 (#48047)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-07-31 03:30:21 +00:00
Jeff (Junze) MaandLucas Wilkinson 80eb01e93d [Bugfix] DSV4 TP16 garbage output (#47493)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-07-07 21:04:33 -07:00
44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-06-22 11:54:14 -07:00
Wentao Ye 0a7bacdcac [DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-06-17 10:55:48 -04:00
Woosuk Kwon f4359a70f9 [DSV4][Minor] Fix supported KV cache dtypes (#44892)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-16 00:14:51 +00:00
Woosuk Kwon 2a983c79ac [DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 (#44699)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-06 20:37:56 -04:00
Woosuk Kwon 4efd6ffde0 [DSV4] Refactor DeepseekV4Attention (#44569)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-06-04 20:23:07 -07:00
Yongye Zhu b5235fca2e [DSv4] Adding TRTLLM gen attention kernel (#43827) 2026-06-04 07:35:09 -07:00