  
|
8a6c0449b3
|
[Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path (#58589)
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-16 21:38:55 +08:00 |
|
 Yongye ZhuandClaude Opus 5.5
|
a6009ea85b
|
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-12 23:24:43 -07:00 |
|
TJian
|
50b21a2667
|
[Perf] [Feat] [ROCm] Add densemha support to ROCm AITER Sparse MLA (#49263)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-10-12 15:50:28 +00:00 |
|
 
|
e572e02d21
|
[UX] Skip checkpoint shards an MTP head does not load (#60152)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 15:16:19 -04:00 |
|
 Qiu ChunshuoandLucas Wilkinson
|
8486f1ebfe
|
[Perf][PCP] Shard decode requests across PCP ranks (#52162)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-01 23:50:13 -04:00 |
|
Wentao Ye
|
4d6874d137
|
[GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-01 15:13:57 +08:00 |
|
 
|
df5668e5c9
|
[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-30 12:19:33 +00:00 |
|
 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
77d9cda688
|
[ROCm][Model][Bugfix] Fix GLM-5.2 shared-expert fusion and MTP on the ROCm DSA path (#58904)
Signed-off-by: Jin Tao <jin.tao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-09-29 07:57:23 +00:00 |
|
Wentao Ye
|
3576691c42
|
[GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-26 11:35:28 -04:00 |
|
  
|
1c0eee919d
|
[ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness (#51915)
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
|
2026-09-22 18:08:51 -05:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)    
|
a7fda4c88b
|
[Bugfix] Fix SM100 fp8_ds_mla cache scales (#49435)
Signed-off-by: Ariel Bereslavsky <arielb@getimpala.ai>
Signed-off-by: Ariel Bereslavsky <ariel@getimpala.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ariel Bereslavsky <arielb@getimpala.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-19 17:15:07 -07:00 |
|
 Julien DebacheandHarry Mellor
|
2bdae2a5a8
|
[fix] Mistral-Large-3 accuracy regression on main (#57563)
Signed-off-by: jdebache <jdebache@nvidia.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-18 14:19:16 +00:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
sychen52
|
7702ee87db
|
[Bugfix][DSA] Write nvfp4_ds_mla from the fused norm+rope kernel (#55538)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-09-15 00:25:33 +04:00 |
|
    
|
ebe1dec2da
|
[PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157)
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-12 23:24:44 +00:00 |
|
      
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
6fe67cbbf3
|
[Spec][V2] Support MTP speculative decoding under pipeline parallelism (#46994)
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 17:55:21 +00:00 |
|
 Yunxiao NingandOpenAI Codex
|
19c018ec05
|
[Bugfix][DCP] Materialize prefill keys on non-owner ranks (#54908)
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-04 02:12:07 +00:00 |
|
 
|
2cf82bcdd1
|
[Bugfix][DCP] Fix NVIDIA DeepSeek-V3.2 / GLM-5.2 fused attention (#50005)
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-08-31 00:30:45 -07:00 |
|
Lucas Wilkinson
|
5707355209
|
[Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA (#54465)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-31 01:13:42 +00:00 |
|
 Matthew BonanniandOpenAI Codex
|
de9250ac9e
|
[Attention] Enable masked MHA for GLM-5 head dimensions (#53785)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-27 21:17:12 +00:00 |
|
 Woosuk KwonandOpenAI Codex
|
c0233fcf01
|
[Model] Remove unused DeepseekV32Indexer forward (#53021)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-19 19:39:52 -07:00 |
|
 Summer YangandWoosuk Kwon
|
58e5ee0158
|
[refactor] consolidate cp attn ops (#52839)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-08-19 18:04:11 -07:00 |
|
Woosuk Kwon
|
b09bd69b5b
|
[Model][NVIDIA] Route DSA models to the CUDA non-compiled path (#52861)
|
2026-08-19 00:13:52 -07:00 |
|
Summer Yang
|
5f7a20b316
|
[nv] add pcp support in dsv3.2 (#52046)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
|
2026-08-18 14:59:08 -07:00 |
|
 Woosuk KwonandOpenAI Codex
|
01e56caaf2
|
[Bugfix][MLA] Do not use Dense MHA for GLM-5.2 (#52512)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-18 11:20:48 -07:00 |
|
fxmarty-amd
|
88b2bff2c6
|
[MOE] Standardize and abstract fused shared expert optimization selection (#51695)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
|
2026-08-18 12:56:56 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) yimdevandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
eab1cff5b0
|
Harden DeepSeek V3.2 fused kernel grids (#52381)
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-18 01:43:12 -07:00 |
|
Jee Jee Li
|
2687fec6ef
|
Replicated embedding and norm fusion for DSV3 flat model (#48484)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-08-18 14:53:21 +08:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
7b544ecb52
|
[Spec Decode][Perf] Fuse the MTP trailing all-reduce; local-argmax draft tokens (#49793)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-14 17:07:07 -07:00 |
|
Woosuk Kwon
|
e644c8cd8c
|
[Perf] Optimize DeepSeek V3.2 sequence parallelism (#51434)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-08-07 16:51:01 -07:00 |
|
Woosuk Kwon
|
46b5864054
|
[Perf] Narrow DeepSeek V3.2 eager CUDA graph region (#51425)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-08-07 14:37:43 -07:00 |
|
Wentao Ye
|
4a11ed5bc4
|
[DSv32/GLM Perf] Skip short prefill topk for dense mha layer, 97.9% kernel level latency reduction (#51298)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-07 20:30:42 +00:00 |
|
Wentao Ye
|
ad5280255b
|
[GLM Perf] DSv32/glm use skip topk for MTP case, 2.0x kernel performance improvement (#50904)
|
2026-08-06 01:48:36 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
2dfb8ba590
|
[Perf][CUDA] Programmatic dependent launch for the DSA decode kernels (#50230)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-06 00:53:17 -07:00 |
|
 xiaozhoupyandClaude
|
3756bf1e2b
|
[Kernel][SM100] Add a CuTeDSL fused query kernel (#49792)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-04 15:23:06 -07:00 |
|
 
|
c810937573
|
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 (#49791)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-03 16:49:41 -07:00 |
|
 Andreas KaratzasandOpenAI Codex
|
f7097a92fd
|
[Bugfix][CI] Prevent common ops imports from initializing CUDA (#50639)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-01 00:13:29 -04:00 |
|
Jee Jee Li
|
92643d68f5
|
K3 DSpark AR fusion (#50242)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-31 11:12:03 +00:00 |
|
 
|
e2efe79695
|
[ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ (#47207)
Signed-off-by: sroberts-amd <sroberts@amd.com>
Signed-off-by: Stacy Roberts <sroberts@amd.com>
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-30 08:37:43 -05:00 |
|
 Yiliu DongandOpenAI Codex
|
ba702e978e
|
[Attention] Skip sparse indexer scoring for dense short prefills (#48407)
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 16:17:22 +00:00 |
|
Wentao Ye
|
7fe6d3c76b
|
[Perf] Fix moe reduce_scatter perf regression by removing additional comm, 5% E2E throughput gain back. (#48763)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-25 16:36:19 +00:00 |
|
Woosuk Kwon
|
213f681f81
|
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768)
|
2026-07-24 16:04:59 -07:00 |
|
     
|
dd72658e7d
|
[Perf][GLM-5.2] Blackwell decode optimizations (#48597)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Signed-off-by: zhou <zhou@zhoudeMacBook-Pro.local>
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zhou <zhou@zhoudeMacBook-Pro.local>
Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com>
|
2026-07-23 21:36:27 -07:00 |
|
 aoshen02andaoshen02
|
ba18929079
|
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-07-21 18:56:39 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
4238b011a7
|
[Feature] Migrate moe sp support to non-torch compiled path for GLM5.2 (#47881)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-15 23:33:15 +00:00 |
|
Wentao Ye
|
1ff9429655
|
[CI Bug] Fully solve accuracy issue for DSv3.2 + MTP + Sequence Parallel (#48036)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-14 10:00:24 -04:00 |
|
 Wei ZhaoandYongye Zhu
|
2afa3f7e95
|
[Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-07-07 21:16:32 -07:00 |
|
Jee Jee Li
|
5d23ca47ab
|
[Kernel] Applies routed_scaling_factor internally (#47408)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-07 02:00:54 -07:00 |
|