 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
b965ee564d
|
[Chore] Use upstream Transformers DeepseekV4Config (#61392)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 01:08:08 -07:00 |
|
 Ronaldandaoshen02
|
19aba3e4d5
|
[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-14 10:17:58 +00:00 |
|
Varun Sundar Rabindranath
|
5463fe4962
|
[KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA (#57652)
|
2026-09-30 09:11:03 -04:00 |
|
 Yongye ZhuandClaude Opus 5.5
|
5ff3bbfb08
|
[Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-24 23:26:50 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Shanshan Shen
|
a6c5d6d0fc
|
[ROCm][Perf] Enable CSA multi-stream overlap for DeepSeek-V4 (#51794)
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
|
2026-09-14 11:21:30 +00:00 |
|
 Roberto L. CastroandCodex
|
120ec4ebd2
|
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels (#53566)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-12 00:05:59 -04:00 |
|
 Roberto L. CastroandCodex
|
6ddbab03de
|
[4/N][warmup][DSv4] Migrate common attention kernels (#50176)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-08 10:25:48 -04:00 |
|
 
|
fc8f10792c
|
Fix DeepSeek V4 FlashMLA auto KV cache dtype (#45091)
Signed-off-by: elehayym <huangxh1618@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: elehayym <huangxh1618@gmail.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-03 19:29:14 +00:00 |
|
Isotr0py
|
1356635d83
|
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-02 11:09:09 -07:00 |
|
 Roberto L. CastroandCodex
|
e16b5e518d
|
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-31 23:26:05 -04:00 |
|
Shanshan Shen
|
32ad1400d7
|
[ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
|
2026-08-27 09:37:54 -07:00 |
|
Woosuk Kwon
|
0a5ad6f0d4
|
[Model] Remove unused DeepSeek V4 top-k buffer helper (#53697)
|
2026-08-26 11:59:09 -07:00 |
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
Fangzhou Ai
|
fe76112ff2
|
[ROCm][Perf] Optimize DeepSeek V4 C4A top-k with AITER (#52882)
|
2026-08-21 10:00:39 -07:00 |
|
 Woosuk KwonandOpenAI Codex
|
f1178f3a06
|
Revert DSv4 eager workspace reuse (#52836)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-18 17:02:59 -07:00 |
|
 
|
292187dd8c
|
[Bugfix][DSv4] Keep indexer scoring in breakable graphs (#52492)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-08-17 03:01:13 +00:00 |
|
 Yongye ZhuandClaude Opus 5
|
967e104fad
|
[Config] Unify indexer cache dtype under attention_config.indexer_kv_dtype (#52550)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-17 10:20:04 +08:00 |
|
Nick Hill
|
8efa13b700
|
[Bugfix] Pick the DeepSeek V4 eager cudagraph region per model runner (#52401)
|
2026-08-15 21:04:08 -07:00 |
|
 Lucas WilkinsonandClaude
|
57bd0ed441
|
[5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-14 09:47:55 -04:00 |
|
 stefankoncarevicandAndreas Karatzas
|
b369f10d5c
|
[Bugfix][ROCm][CI] Restore the DeepSeek-V4 input GEMM override point (#51821)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-12 20:33:52 -05:00 |
|
  ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)   
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
 Woosuk KwonandOpenAI Codex
|
79c865b838
|
[Perf] Narrow DeepSeek V4 eager CUDA graph region (#51430)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-10 10:52:10 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
df71917cf1
|
[DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. (#49236)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-31 08:45:58 -07:00 |
|
 
|
12a34a6bc7
|
[ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-30 13:59:52 -05:00 |
|
+13        
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
Wentao Ye
|
b0cb1da1bd
|
[DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case (#49486)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-23 13:08:08 -04:00 |
|
Summer Yang
|
f3a920a076
|
[Core][DSV4] Compact MXFP4 indexer KV cache and packed group overlays (#48993)
|
2026-07-22 16:58:44 -07:00 |
|
 Jeff (Junze) MaandLucas Wilkinson
|
80eb01e93d
|
[Bugfix] DSV4 TP16 garbage output (#47493)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-07-07 21:04:33 -07:00 |
|
 
|
04adc8843b
|
[Bugfix]Fix DeepSeek-V4 fp8_ds_mla KV cache reshape (#47716)
Co-authored-by: yy-fighting <23518844576@qq.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-07-06 12:56:44 -07:00 |
|
 Kunshang JiandHarry Mellor
|
07f9baf756
|
Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-06 14:18:33 +01:00 |
|
 Kunshang JiandHarry Mellor
|
e840f0d3f5
|
[Platform] Replace torch.cuda.Event with torch.Event (#47140)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-30 08:39:59 -07:00 |
|
 
|
44d95069e9
|
Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-06-22 11:54:14 -07:00 |
|
 Woosuk KwonandClaude Opus 4.8
|
1797576237
|
Revert "[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture" (#45309) (#45972)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-17 17:20:59 -07:00 |
|
Wentao Ye
|
2a47a9ff0f
|
[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 09:34:53 -07:00 |
|
Woosuk Kwon
|
02d2da0748
|
[DSV4] Move more ops out of eager breakpoint (#44561)
|
2026-06-05 06:42:41 -07:00 |
|
Woosuk Kwon
|
4efd6ffde0
|
[DSV4] Refactor DeepseekV4Attention (#44569)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-04 20:23:07 -07:00 |
|
Yongye Zhu
|
b5235fca2e
|
[DSv4] Adding TRTLLM gen attention kernel (#43827)
|
2026-06-04 07:35:09 -07:00 |
|
Woosuk Kwon
|
8c3cc98cff
|
[DSV4] Remove unncessary classes & functions (#44246)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-01 14:43:00 -07:00 |
|
Woosuk Kwon
|
69b8956dcd
|
[Model Refactoring] Remove unncessary torch op registration for DSv4 (#43891)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-05-28 14:04:55 -07:00 |
|
Yongye Zhu
|
6ab6ffb428
|
[Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162)
|
2026-05-26 05:12:54 -10:00 |
|
Yongye Zhu
|
843715739b
|
[Refactor] Extract DeepSeek V4 sparse MLA impl into model folder (#43149)
|
2026-05-22 10:06:31 -07:00 |
|
Woosuk Kwon
|
b14be81c1f
|
[Model Refactoring] Move deepseek_v4_ops to models/deepseek_v4 [3/N] (#43073)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-05-19 00:52:54 -07:00 |
|
Woosuk Kwon
|
87b08c5f64
|
[Model Refactoring] Move DeepSeek V4 layers to models/deepseek_v4/ [2/N] (#43039)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-05-18 21:00:58 -07:00 |
|