Yongye Zhu and Claude Opus 5.5
5ff3bbfb08
[Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA ( #58621 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 23:26:50 -07:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00
Jhao-Ting Chen
4fb767ff0a
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection ( #53175 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
2026-09-23 20:29:06 -07:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Isotr0py
1356635d83
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support ( #54566 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-02 11:09:09 -07:00
Ilya Markov
25efcfa788
[Attention] Enable adaptive verification for FLASHINFER_MLA_SPARSE_DSV4 ( #52724 )
...
Signed-off-by: Markov Ilya <markovilya197@gmail.com >
2026-09-01 13:43:28 +00:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
Woosuk Kwon and OpenAI Codex
f1178f3a06
Revert DSv4 eager workspace reuse ( #52836 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 17:02:59 -07:00
Gabriel Wu
97388c44f9
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark ( #51538 )
2026-08-15 07:01:32 -07:00
7f7a32cfec
[Spec Decode] DSpark confidence-scheduled verification ( #47808 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-12 12:27:48 -07:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
df71917cf1
[DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. ( #49236 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
b49eaf205a
[DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 ( #48047 )
...
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-07-31 03:30:21 +00:00
Jeff (Junze) Ma and Lucas Wilkinson
80eb01e93d
[Bugfix] DSV4 TP16 garbage output ( #47493 )
...
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-07-07 21:04:33 -07:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
Wentao Ye
0a7bacdcac
[DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement ( #45863 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-17 10:55:48 -04:00
Woosuk Kwon
f4359a70f9
[DSV4][Minor] Fix supported KV cache dtypes ( #44892 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-06-16 00:14:51 +00:00
Woosuk Kwon
2a983c79ac
[DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 ( #44699 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-06-06 20:37:56 -04:00
Woosuk Kwon
4efd6ffde0
[DSV4] Refactor DeepseekV4Attention ( #44569 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-06-04 20:23:07 -07:00
Yongye Zhu
b5235fca2e
[DSv4] Adding TRTLLM gen attention kernel ( #43827 )
2026-06-04 07:35:09 -07:00