 Kevin H. LuuandMichael Goin
|
5f8f72866c
|
[Build] Fix CUDA release wheel builds (#50243)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-07-30 11:48:09 -07:00 |
|
|
MollyandMichael Goin
|
451227cb3f
|
[Bugfix][Kernel] Fix integer overflow in libtorch_stable/activation_kernels.cu (#49660)
Signed-off-by: molly-ting <molly.cn.mail@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-07-29 17:50:44 -07:00 |
|
|
Wentao Ye
|
82642d7d6c
|
[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-07-29 13:45:38 -07:00 |
|
|
+12
|
7c6729b769
|
[Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-07-29 14:10:58 +08:00 |
|
|
|
b6cbba8bc8
|
[Bugfix][Kernel] Fix batch invariance in RMSNorm kernels by pinning block size (#48391)
Signed-off-by: oops-oom <73481342@qq.com>
Signed-off-by: oops-oom <liubin8905@vip.qq.com>
Co-authored-by: oops-oom <73481342@qq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-07-28 22:24:02 +08:00 |
|
|
Thien Tran
|
61ac368021
|
[Kimi-K3] Add AttnRes kernels (#50090)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-07-28 00:22:09 -07:00 |
|
|
Woosuk Kwon
|
213f681f81
|
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768)
|
2026-07-24 16:04:59 -07:00 |
|
|
|
dd72658e7d
|
[Perf][GLM-5.2] Blackwell decode optimizations (#48597)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Signed-off-by: zhou <zhou@zhoudeMacBook-Pro.local>
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zhou <zhou@zhoudeMacBook-Pro.local>
Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com>
|
2026-07-23 21:36:27 -07:00 |
|
|
Nick Hill
|
910cc8543a
|
[Bugfix] Restore gather_and_maybe_dequant_cache OOB guard (#49427)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-22 13:47:37 -07:00 |
|
|
gnovack
|
85f638a2b8
|
skip cudagraph/DP padding in topk (#48979)
Signed-off-by: gnovack <novackgm@gmail.com>
|
2026-07-21 15:20:30 -07:00 |
|
|
|
0d9210a502
|
Fixes non-coalesced HBM access in marlin_int4_fp8_preprocess_kernel_awq (#47268)
Signed-off-by: xjx <493337577@qq.com>
Signed-off-by: flutist-alibaba <30485581+flutist@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-07-20 18:30:38 -07:00 |
|
|
Wentao Ye
|
b5433b6f50
|
[Perf] Optimize dsv4 routing using specialized kernel, 2.94% E2E TPOT improvement (#48660)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-17 13:35:06 -07:00 |
|
|
|
67f9046e4a
|
[Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-16 22:44:04 +00:00 |
|
|
Wentao Ye
|
2dab187f75
|
[Perf] Optimize fused_topk_bias for DSv4, 1.5~2x kernel performance improvement (#47463)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-15 23:40:55 +00:00 |
|
|
gnovack
|
f7aadae5e5
|
add pad-aware reduce path (#48385)
Signed-off-by: gnovack <novackgm@gmail.com>
|
2026-07-14 18:05:50 -07:00 |
|
|
|
ee5a89f4d7
|
[ROCm][MiniMax-M3] Add AITER sparse paged attention (#47287)
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Jun Kang Chow <junkangchow@gmail.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-07-12 19:27:29 -07:00 |
|
|
Tanish MalekarandClaude Sonnet 4.6
|
e26264f3ef
|
[Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) (#39058)
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-07-12 19:18:03 -07:00 |
|
|
|
51878e5b6e
|
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends (#44455)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-07-11 11:11:16 -04:00 |
|
|
|
19069bcbd5
|
FP32 router GEMV optimization (#48335)
Signed-off-by: peiyuanz <peiyuanz@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: peiyuanz <peiyuanz@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: zhouzhou <zhouzhou@zhouzhoudeMacBook-Pro.local>
|
2026-07-11 13:07:48 +00:00 |
|
|
gnovack
|
f378f79b7c
|
handle topk_ids padding in align sum kernel (#47785)
Signed-off-by: gnovack <novackgm@gmail.com>
|
2026-07-10 13:33:28 -07:00 |
|
|
Michael Goin
|
08dfd68610
|
[Model] Add LongCat-Flash-Lite (n-gram embedding) (#47857)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-10 07:17:50 -07:00 |
|
|
Jee Jee Li
|
5d23ca47ab
|
[Kernel] Applies routed_scaling_factor internally (#47408)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-07 02:00:54 -07:00 |
|
|
Chris Leonard
|
fbc9ba6d30
|
New stable abi cleanup (#46656)
Signed-off-by: Chris Leonard <chleonar@redhat.com>
|
2026-07-03 14:02:26 +08:00 |
|
|
Michael Goin
|
d715b3aa1e
|
Delete PagedAttention (#47361)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-02 12:31:26 -07:00 |
|
|
TJian
|
de2a8fc042
|
[ROCm] [PyTorch] Move to stable abi since ROCm upgraded to torch 2.11 (#47128)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-07-02 18:34:07 +08:00 |
|
|
Jee Jee Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d0a2584773
|
[Misc] Use functions instead of PTX for the PDL instruction (#46984)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-01 19:38:35 -07:00 |
|
|
Wentao Ye
|
9e84ec8648
|
[Refactor] Remove dead minimax allreduce rms kernel (#46842)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-30 08:29:21 -07:00 |
|
|
Matt
|
1a4984520e
|
[Hardware][AMD][CI] Fix AMD CI image build (#46792)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-06-25 22:05:12 -07:00 |
|
|
Giancarlo Delfinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
02a1f23711
|
[DFlash] Fuse precompute kv per-layer rmsnorms (#46761)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-25 19:32:07 -07:00 |
|
|
Michael GoinandClaude
|
e8c24a7695
|
[Kernel] Vectorized fp32 moe_sum reduction and support any topk (#46643)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-25 14:02:28 -07:00 |
|
|
Jee Jee Li
|
23aed9b0ee
|
[Kernel] Enable PDL for per_token_group_quant_8bit_kernel (#46508)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-25 08:42:51 +08:00 |
|
|
Cyrus Leung
|
24d5186138
|
[Bugfix] Re-enable FP8 MoE on NVIDIA Thor (#46339)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-06-24 07:35:46 -07:00 |
|
|
Mohammad Miadh Angkad
|
191826ec61
|
[CI/Build] Fix topk histogram build on SM75 (#46550)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
|
2026-06-24 00:51:11 -07:00 |
|
|
Jee Jee Li
|
9d6fdc2901
|
[Kernel] GLM5 Router GEMM (#46385)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-23 22:54:50 -07:00 |
|
|
Roberto L. Castro
|
855cd4d787
|
[Perf][DSv4/DSv3.2] Add cluster-cooperative topK kernel for low-latency scenarios (#43008)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
|
2026-06-23 16:11:00 -07:00 |
|
|
|
6691f087a6
|
[Minimax-M3] BF16/FP8 Indexer using MSA (#45892)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-06-23 10:28:49 -07:00 |
|
|
|
7d47cff933
|
[Bugfix][KV Offload] Fix swap_blocks_batch on the default stream (#46379)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-23 05:45:27 -07:00 |
|
|
Yifan Qiao
|
aa4990a9a2
|
[Attention] Re-enable cross-layer KV cache layout for MLA via stride-aware kernels (#45111)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-22 06:57:02 -07:00 |
|
|
|
ebfbcfe46a
|
Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg (#45026)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com>
|
2026-06-20 13:38:10 -07:00 |
|
|
JasonLi314
|
93bad11912
|
[Bugfix] Fix gridDim.y overflow for large row counts (#45255)
Signed-off-by: Jason Li <li.jason.cs@gmail.com>
|
2026-06-19 23:27:45 -04:00 |
|
|
Chris Leonard
|
b9a7cd464c
|
[12/n] final _C library kernel migration (#45415)
|
2026-06-19 06:57:26 -07:00 |
|
|
Wentao Ye
|
225936a1dd
|
[CI Bug] Revert #42379 to fix CI Multi-Modal Models (Extended Generation 1) (#46070)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-18 12:37:39 -07:00 |
|
|
Humphrey
|
4583630b56
|
[Bugfix][Kernel] Check output alignment in vectorize_with_alignment (fixes misaligned-address crash for non-multiple-of-8 head sizes) (#45466)
Signed-off-by: HumphreySun98 <humphreysun98@gmail.com>
|
2026-06-18 16:58:22 +00:00 |
|
|
Jee Jee Li
|
22cc891108
|
[Kernel] Add PDL support for DeepGEMM kernel (#46006)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-18 20:49:01 +08:00 |
|
|
|
afdcbd5d39
|
[ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681)
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
|
2026-06-18 12:21:14 +00:00 |
|
|
Micah Williamson
|
e945169207
|
Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999)
|
2026-06-17 22:59:48 -07:00 |
|
|
Wentao YeandShengqi Chen
|
b4c80ec0fd
|
[Refactor] Remove dead cutlass mxfp8 code (#44681)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-06-17 21:18:25 -07:00 |
|
|
Jee Jee Li
|
4403af8fb5
|
[Kernel] Add PDL support for DeepGEMM kernel (#42996)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-17 20:37:17 -07:00 |
|
|
arghyadeep sarkar
|
93bbe94d3a
|
[Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False (#41430) (#44109)
Signed-off-by: hello-args <args.sarkar@gmail.com>
|
2026-06-16 23:45:55 -07:00 |
|
|
Thien TranandYongye Zhu
|
4c62663315
|
[M3] Enable FP8 sparse GQA (#45744)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-06-16 21:38:03 -07:00 |
|