 Harry MellorandClaude Opus 5
|
78300cdabf
|
[Bugfix][Docs] Package glm5next nvidia subtree and fix its docstrings (#55214)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-03 21:25:26 -07:00 |
|
|
aoshen02andaoshen02
|
69cf055936
|
[Performance][DSv4] Size dequant gather launch grid by rows (#55061)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
|
2026-09-03 20:06:34 -07:00 |
|
|
Yunxiao NingandOpenAI Codex
|
19c018ec05
|
[Bugfix][DCP] Materialize prefill keys on non-owner ranks (#54908)
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-04 02:12:07 +00:00 |
|
|
Yongye ZhuandClaude Fable 5
|
bc2ee48073
|
[Perf] Prefetch the weight before the PDL wait in fused_q_kv_rmsnorm (#55020)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-03 15:40:16 -07:00 |
|
|
|
cee0f92c02
|
[ROCm][Perf] Optimize MiniMax-M3 decode indexer and top-k (#54682)
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-03 13:20:18 -07:00 |
|
|
|
fc8f10792c
|
Fix DeepSeek V4 FlashMLA auto KV cache dtype (#45091)
Signed-off-by: elehayym <huangxh1618@gmail.com>
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: elehayym <huangxh1618@gmail.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-03 19:29:14 +00:00 |
|
|
sychen52andVadim Gimpelson
|
d4d703caf9
|
[Bugfix][Model] Fix FP8 PLE loading in mixed ModelOpt checkpoints (#54882)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
|
2026-09-03 21:09:39 +04:00 |
|
|
Bolin SunandClaude Fable 5
|
d410fc12f3
|
[Kernel] Enable Kimi-K3 SiTU on the CuteDSL MoE backend and the SM107 low-latency GEMM plan (#54606)
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-03 16:49:31 +00:00 |
|
|
Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Wentao Ye
|
a56654d6de
|
[K3 Perf] Enable DSV3 GEMM for inner-contiguous and row-strided tensors, 12%~81% kernel performance improvement (#54565)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-02 12:22:40 -07:00 |
|
|
Isotr0py
|
1356635d83
|
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-02 11:09:09 -07:00 |
|
|
Thien TranandCodex
|
f870b92976
|
[Qwen3.8-Flash-Next] Fuse Qwen4Exp PLE kernels (#54517)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-02 12:32:16 +08:00 |
|
|
Harry Huang
|
1e300895ac
|
[Qwen4] validate FP8 PLE weight scale after loading (#54722)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
|
2026-09-02 04:30:58 +00:00 |
|
|
Isotr0py
|
1d8d7a3965
|
[Bugfix] Fix RoPE construction for deepseek-v4 sparse SWA layers (#54815)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-01 20:48:50 -07:00 |
|
|
Yongye ZhuandOpenAI Codex
|
3ba9907a1d
|
[Kimi-K3] Overlap low-M TP8 KDA projections (#54697)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-02 03:35:28 +00:00 |
|
|
Thien Tran
|
003e34341a
|
[Qwen3.8-Flash-Next] Separate prefill and decode paths for QSA indexer (#54513)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-09-02 09:08:10 +08:00 |
|
|
Wentao Ye
|
fc72fc39ac
|
[Kimi Bug] Fix cannot access local variable 'active_non_spec_mask_cpu' (#54781)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-01 21:08:50 +00:00 |
|
|
Ilya Markov
|
25efcfa788
|
[Attention] Enable adaptive verification for FLASHINFER_MLA_SPARSE_DSV4 (#52724)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
|
2026-09-01 13:43:28 +00:00 |
|
|
Zheng Cai
|
191cecd51e
|
[Kernel][Qwen] Add Hopper LL-GEMM tuning table for Qwen4Exp (#54560)
Signed-off-by: Zheng Cai <8370601+zigzagcai@users.noreply.github.com>
|
2026-09-01 11:21:41 +00:00 |
|
|
Roberto L. CastroandCodex
|
e16b5e518d
|
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-31 23:26:05 -04:00 |
|
|
Yongye Zhu
|
d6d6658543
|
[Kimi-K3][Perf] Make native CUDA AttnRes the SM100 default (#54261)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-08-31 23:18:12 +00:00 |
|
|
|
699e180df4
|
[Bugfix][SM120] DSv4: pass contiguous C128A decode topk indices on SM120 (#53574)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-31 09:44:42 +00:00 |
|
|
|
2cf82bcdd1
|
[Bugfix][DCP] Fix NVIDIA DeepSeek-V3.2 / GLM-5.2 fused attention (#50005)
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-08-31 00:30:45 -07:00 |
|
|
|
da0b2d8b17
|
[Performance] Optimize Dots3 NOTE runtime (#53517)
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-31 00:05:18 -07:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
Lucas Wilkinson
|
5707355209
|
[Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA (#54465)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-31 01:13:42 +00:00 |
|
|
Isotr0py
|
4fc943b867
|
[Multimodal] Deprecate PyAV video decoder backend (#54231)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-08-29 15:13:28 +00:00 |
|
|
|
b2f685834a
|
[Hy4] support Hy4-preview model (#54160)
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-29 12:07:28 +08:00 |
|
|
|
ae5b8e4a8d
|
[ROCm][PERF] Enable AITER PA gluon decode for MiniMax-M3 MTP and dense layers (#52849)
Signed-off-by: ukannika <uma.kannikanti@amd.com>
Signed-off-by: ykamiset <yaminipreethi.kamisetty@amd.com>
Co-authored-by: ykamiset <yaminipreethi.kamisetty@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-28 22:54:39 +00:00 |
|
|
Yongye ZhuandOpenAI Codex
|
d9dabfa351
|
[Kimi-K3][Kernel] Optimize the low-M fused latent MoE tail (#54168)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-28 10:45:36 -07:00 |
|
|
Thien Tran
|
f956e1c343
|
[Kimi-K3][Bugfix] Fix low-latency GEMM fallback initialization (#54167)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-08-28 08:32:20 +00:00 |
|
|
Roy WangandClaude Fable 5
|
b74ac0bc61
|
[Bugfix] Allocate packed outputs in fused_q_kv_rmsnorm so q_b_proj keeps its low-latency GEMM path at decode (#53109)
Signed-off-by: yasong.wang <yasong.wang@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-28 02:08:27 +00:00 |
|
|
Thien Tran
|
6ec92bcbc8
|
[Kimi-K3] Merge MLA gate into QKV-A projection (#54015)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-08-27 17:47:18 -07:00 |
|
|
Matthew BonanniandOpenAI Codex
|
de9250ac9e
|
[Attention] Enable masked MHA for GLM-5 head dimensions (#53785)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-27 21:17:12 +00:00 |
|
|
Wentao Ye
|
9818bb3db8
|
[Kimi Perf] Tune hopper low latency gemm kernel, 4%~97% performance improvement (#54088)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-27 16:48:53 -04:00 |
|
|
Shanshan Shen
|
32ad1400d7
|
[ROCm][Perf] Fuse SWA q/kv RMSNorm and q FP8 group quant for DeepSeek-V4 (#53540)
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
|
2026-08-27 09:37:54 -07:00 |
|
|
Canlin Guo
|
aa640684cc
|
[Kimi K3][Kernel] Support DS conv-state layout in fused KDA decode kernel (#53396)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-08-26 23:54:50 -07:00 |
|
|
Wentao Ye
|
76cfe1cd88
|
[Kimi K3 Perf] Optimize eh_proj linear calculation, 12.9 ~ 25.2% kernel performance improvement (#53942)
|
2026-08-26 17:08:02 -04:00 |
|
|
Woosuk Kwon
|
0a5ad6f0d4
|
[Model] Remove unused DeepSeek V4 top-k buffer helper (#53697)
|
2026-08-26 11:59:09 -07:00 |
|
|
Fangzhou Ai
|
657f9b9ce2
|
[ROCm][DSV4][Perf] Fuse DeepSeek V4 C4 compressor GEMMs (#53838)
Signed-off-by: fai <fangzhouai@gmail.com>
|
2026-08-26 23:36:14 +08:00 |
|
|
Canlin Guo
|
bc2d63e650
|
[Kimi K3][Kernel] Enable low-latency decode GEMM dispatch on SM100 (#53534)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-08-25 16:32:28 -07:00 |
|
|
Wentao Ye
|
41729fc53b
|
[K3 Perf] Optimize k3 mamba metadata preparation, 6.6~7.6x kernel performance improvement (#52388)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-25 11:00:28 -04:00 |
|
|
|
8fe9317f2e
|
[Model][MoE] DeepSeek-V4: add opt-in FlashInfer moe_ep expert backend (#49636)
Signed-off-by: Md Saidul Hoque Anik <mhoqueanik@nvidia.com>
Signed-off-by: Anerudhan Gopal <agopal@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
|
2026-08-25 03:35:01 +00:00 |
|
|
Giancarlo Delfin
|
d154d90d6c
|
[Model Runner V2] batch-sharded sample (#50465)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-24 15:49:57 -07:00 |
|
|
Kevin H. LuuandCodex
|
4c56e62c85
|
[Bugfix][Kimi K3] Skip absent metadata during CUDA graph profiling (#53581)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-24 15:16:28 +00:00 |
|
|
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟andJee Jee Li
|
702e1d7186
|
[LoRA] feat: Support LoRA for DeepSeek V4 (#53361)
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-24 03:57:57 +00:00 |
|
|
范裕达
|
185cada36b
|
[Model] Fix KV cache layout and optimize Dots3 NOTE Omni encoders (#53460)
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
|
2026-08-23 10:02:35 -07:00 |
|
|
Cyrus LeungandQwen Code
|
ebcd606467
|
[MM] Simplify prompt updates: replace PromptSeq with list[int] (#53372)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Qwen Code <noreply@alibaba-inc.com>
|
2026-08-23 14:50:32 +08:00 |
|
|
|
9eb9d9d395
|
[Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement (#52789)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-22 10:32:40 -07:00 |
|
|
Cyrus Leung
|
0b19ebcacd
|
[MM] Simplify _apply_hf_processor_main (#53275)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-08-22 01:49:37 -07:00 |
|