 Woosuk KwonandCodex
|
9a70c233cd
|
[Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-20 20:28:06 -07:00 |
|
|
Wentao Ye
|
4868312128
|
[Refactor] Remove dead kernel code (#57621)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-20 10:39:23 -07:00 |
|
|
|
7d99c2c4fd
|
[Feature][Humming] Humming feature integration (#56685)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-20 08:11:01 -07:00 |
|
|
|
a7fda4c88b
|
[Bugfix] Fix SM100 fp8_ds_mla cache scales (#49435)
Signed-off-by: Ariel Bereslavsky <arielb@getimpala.ai>
Signed-off-by: Ariel Bereslavsky <ariel@getimpala.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ariel Bereslavsky <arielb@getimpala.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-19 17:15:07 -07:00 |
|
|
Thien TranandCodex
|
8da75d6190
|
C3x SM100 FP8 blockwise - Pad activation scales to multiple of 4 (#57377)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-18 08:39:27 +00:00 |
|
|
Wentao Ye
|
91a4c40b45
|
[Compile] Fix compile warning #177-D (#56609)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-17 05:04:56 -07:00 |
|
|
|
d6a1677d55
|
[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Yunxiao Ning <73625538+foraxe@users.noreply.github.com>
Co-authored-by: AI Assistant <noreply@openai.com>
|
2026-09-16 03:07:50 -07:00 |
|
|
|
6ecd97f1b7
|
[Perf][DSpark] Add KV-only context insertion across V4.1 cache formats (#56441)
Signed-off-by: 0z5a <0z5a@users.noreply.github.com>
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: 0z5a <0z5a@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 21:34:55 -07:00 |
|
|
Misha GoinandOpenAI Codex
|
c6fa1f05d1
|
[Perf][Kernel] Add sampled filtering for persistent top-k (#56346)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-15 14:15:59 -07:00 |
|
|
Fangzhou Ai
|
dffbb714e4
|
[ROCm][Perf] Optimize DSV4.1 K=512 decode top-k on gfx950 (#56743)
Signed-off-by: fai <fangzhouai@gmail.com>
|
2026-09-15 18:34:24 +00:00 |
|
|
|
e6eb0d120c
|
[Model][DSv4.1] Store the whole KV in MXFP8 (FlashMLA V4.1 record) (#56893)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-14 20:12:55 -07:00 |
|
|
sychen52
|
7702ee87db
|
[Bugfix][DSA] Write nvfp4_ds_mla from the fused norm+rope kernel (#55538)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-09-15 00:25:33 +04:00 |
|
|
Ayushman SinghandAyushman Singh
|
c2ea9f1a5f
|
[Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization (#56478)
Signed-off-by: Ayushman Singh <ayush1399@users.noreply.github.com>
Co-authored-by: Ayushman Singh <ayush1399@users.noreply.github.com>
|
2026-09-12 20:27:08 -07:00 |
|
|
|
29332cf936
|
[4/N] Expose HiSparse cache metrics via KV connector stats (#56061)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 10:47:55 -07:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
Misha Goin
|
8a7f98c8c3
|
[Kernel][MoE] Optimize batched_moe_align_block_size with cooperative writes (#53280)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-11 12:05:22 -07:00 |
|
|
Wentao Ye
|
1e1060f998
|
[Kimi Perf] Group fp8 mla cahche insertion, 4~6x kernel level performance improvement for small batch (#55356)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-11 08:16:57 -07:00 |
|
|
|
9521c60bdc
|
[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill reland (#54038)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <17350011+kliuae@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-09-10 15:37:54 +08:00 |
|
|
Yongye ZhuandClaude Opus 5
|
be1cb9834b
|
[Kernel] Optional Q-norm in fused DSv4 MLA epilogue; group_size=32 for packed FP8 quant (#56215)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-10 00:35:59 -07:00 |
|
|
Roderick Wuandmgoin
|
2c9d68f80b
|
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809)
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-08 12:45:12 -07:00 |
|
|
Jee Jee LiandYongye Zhu
|
9c297e3b8b
|
[Kernel] PDL enablement for fusedQKNormRopeKernel (#55755)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-07 20:49:03 -07:00 |
|
|
Jiangyun Zhu
|
f43ef1531d
|
[CI] fix pre-commit (#55630)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-09-06 20:54:22 -07:00 |
|
|
Jürgen Schmied
|
4df80187b5
|
[Kernel] SM 12.x blockwise FP8: swizzle the CTA raster when the weight exceeds the L2 (#55180)
|
2026-09-06 17:36:59 -07:00 |
|
|
lucamotzandLuca Motz
|
bc96d76aa9
|
[Kernel] Fall back from persistent top-k on low-shared-memory GPUs (#54110)
Signed-off-by: Luca Motz <321921718+lucamotz@users.noreply.github.com>
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Luca Motz <321921718+lucamotz@users.noreply.github.com>
|
2026-09-05 08:58:37 -04:00 |
|
|
Yongye ZhuandClaude Fable 5
|
9509fc8ae6
|
[Perf][Kimi-K3] Cut MLA decode concat/cache epilogue latency (#54896)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-03 15:39:58 -07:00 |
|
|
Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Wentao Ye
|
a56654d6de
|
[K3 Perf] Enable DSV3 GEMM for inner-contiguous and row-strided tensors, 12%~81% kernel performance improvement (#54565)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-02 12:22:40 -07:00 |
|
|
Isotr0py
|
1356635d83
|
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support (#54566)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-02 11:09:09 -07:00 |
|
|
|
46c8a161f4
|
[Bugfix] Handle padded routes in CUTLASS MoE permutations (#54747)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-01 19:08:17 -07:00 |
|
|
|
514c7314a0
|
[Perf][Kernel] Initialize NVFP4 padding in quant kernel (#53568)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-01 09:49:07 -07:00 |
|
|
Misha GoinandOpenAI Codex
|
82b7d49a6e
|
[MoE] Generalize masked activation for padded layouts (#51217)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-01 11:34:13 -04:00 |
|
|
sychen52
|
7c5dc571cb
|
[Attention][DSA] Enable W4A16 DSA (#51724)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-09-01 12:37:47 +08:00 |
|
|
Yongye Zhu
|
d6d6658543
|
[Kimi-K3][Perf] Make native CUDA AttnRes the SM100 default (#54261)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-08-31 23:18:12 +00:00 |
|
|
|
f5c3cc240b
|
[Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-31 13:44:38 -07:00 |
|
|
Jee Jee Li
|
e2c8eeac40
|
[kernel] Fused embedding kernel (#53677)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-08-31 11:34:14 +00:00 |
|
|
Jee Jee Li
|
8e92248f79
|
[Kernel] Retire the DSv3 router GEMM CUDA kernel (#54040)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-08-31 07:15:31 +00:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
|
06cccf8730
|
[Bugfix] Fix int32 token offset overflow in fused SiLU block quant (#53409)
Signed-off-by: Anni <dranox2222@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-08-28 16:23:36 -04:00 |
|
|
Misha GoinandOpenAI Codex
|
5bbc58c0ad
|
[Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-27 19:01:22 -07:00 |
|
|
prakharPant
|
b2a6e9dca0
|
fix(build): correct preprocessor guard for GDN decode to fix Ampere c… (#52743)
Signed-off-by: prakharP <prakharpant288@gmail.com>
|
2026-08-27 17:21:44 -07:00 |
|
|
Canlin Guo
|
aa640684cc
|
[Kimi K3][Kernel] Support DS conv-state layout in fused KDA decode kernel (#53396)
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-08-26 23:54:50 -07:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
a5e0045433
|
[Refactor] Remove dead code quantization 2 (#52821)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-25 20:28:44 -07:00 |
|
|
|
ca29cd48ba
|
[Quantization][Humming] Support MXFP4 weight + block-FP8 activation for MoE (#51332)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-25 10:21:00 -04:00 |
|
|
Kevin H. LuuandAndreas Karatzas
|
592e06f2ae
|
Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill" (#53294)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-21 13:34:13 -07:00 |
|
|
Fangzhou Ai
|
fe76112ff2
|
[ROCm][Perf] Optimize DeepSeek V4 C4A top-k with AITER (#52882)
|
2026-08-21 10:00:39 -07:00 |
|
|
|
463aa5e30f
|
[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill (#52606)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-08-21 09:07:13 -07:00 |
|
|
Hongxia YangandYifan Qiao
|
cd7b7c265a
|
[ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-21 09:07:03 -07:00 |
|
|
Gabriel Wu
|
2b7fcbf527
|
[Kernel] SM120: stop routing misaligned-M blockwise FP8 GEMMs to the small-M swapAB config (#52775)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
|
2026-08-19 21:25:29 +08:00 |
|
|
Woosuk KwonandOpenAI Codex
|
f1178f3a06
|
Revert DSv4 eager workspace reuse (#52836)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-18 17:02:59 -07:00 |
|
|
kiroxu
|
cdb8545a91
|
[Kernel][Perf] Support Qwen head ratios in fused GDN MTP (#52539)
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com>
|
2026-08-17 18:17:40 -07:00 |
|