Commit Graph
308 Commits
Author SHA1 Message Date
gnovack f7aadae5e5 add pad-aware reduce path (#48385)
Signed-off-by: gnovack <novackgm@gmail.com>
2026-07-14 18:05:50 -07:00
Daoyuan LiWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ca3618bc69 [Doc] Sync four function docstrings with their signatures (#45437)
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-14 13:10:13 -04:00
Matthias GehreandAndreas Karatzas 382bbd5144 [ROCm][Kernel] Add HybridW4A16LinearKernel: Triton prefill + HIP skinny decode (#40977)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-07-13 20:22:00 -07:00
guybdCursorLi, Jiang <jiang1.li@intel.com>
05fa8183a6 [CPU][Spec Decode] Support DFlash speculative decoding for GDN models on CPU (#46090)
Signed-off-by: guybd <guy.boudoukh@intel.com>
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-07-13 04:16:18 +00:00
ee5a89f4d7 [ROCm][MiniMax-M3] Add AITER sparse paged attention (#47287)
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Jun Kang Chow <junkangchow@gmail.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
2026-07-12 19:27:29 -07:00
Qiming ZhangandKunshang Ji 0b6636cbcb [XPU]remove is_xxx from moe class and bump up kernels (#48079)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-07-11 09:27:13 +00:00
Michael Goin 08dfd68610 [Model] Add LongCat-Flash-Lite (n-gram embedding) (#47857)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-07-10 07:17:50 -07:00
liuzhenwei b93cbd7416 [XPU] Fix topk_sigmoid arg mismatch on XPU (#47858)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
2026-07-07 16:53:18 +00:00
Jee Jee Li 5d23ca47ab [Kernel] Applies routed_scaling_factor internally (#47408)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-07-07 02:00:54 -07:00
Fadi Arafeh f1073c050c [CPU][BugFix] Multiple fixes to w4a8_int8 CPU MoE path (#46739)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
2026-07-06 05:39:20 +00:00
Michael Goin d715b3aa1e Delete PagedAttention (#47361)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-07-02 12:31:26 -07:00
Albert ChengandVadim Gimpelson 92c7fac640 [Perf] Restore zero-init of swizzled NVFP4 scale buffer to recover Blackwell decode throughput (#45739)
Signed-off-by: Albert Cheng <albertching0112@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
2026-06-30 22:56:56 +00:00
Wentao Ye 9e84ec8648 [Refactor] Remove dead minimax allreduce rms kernel (#46842)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-06-30 08:29:21 -07:00
Wentao Ye cc7981599e [Refactor] Remove dead kernel code (#46405)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-06-25 18:09:56 -07:00
Wentao YeandShengqi Chen b4c80ec0fd [Refactor] Remove dead cutlass mxfp8 code (#44681)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
2026-06-17 21:18:25 -07:00
arghyadeep sarkar 93bbe94d3a [Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False (#41430) (#44109)
Signed-off-by: hello-args <args.sarkar@gmail.com>
2026-06-16 23:45:55 -07:00
Thien TranandYongye Zhu 4c62663315 [M3] Enable FP8 sparse GQA (#45744)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-06-16 21:38:03 -07:00
Li, Jiang a7fdfeef72 [CPU] Support Gemma Diffusion (#45690)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-06-16 14:39:56 +08:00
0a1c5034f5 [Model] Add MiniMax M3 support (#45381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-06-16 01:01:25 +08:00
78e7293bb1 [Build] Fix CUDA arch build coverage gaps (#45277)
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Xin Li <xinli-sw@users.noreply.github.com>
Co-authored-by: ShawRong <ShawRong@users.noreply.github.com>
Co-authored-by: Change72 <Change72@users.noreply.github.com>
2026-06-13 22:09:20 -07:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-12 12:02:21 -07:00
Li, Jiang 980796cd07 [CI/Build][CPU] Fix flaky CI image build failure and unexpected warnings (#44852)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-06-08 11:10:06 +00:00
Chaojun Zhang 810966453a [XPU] Support cpu kv offloading and tiering offloading on XPU platform (#36423)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
2026-06-07 09:59:28 +08:00
JartXandTJian 062b05ff3a [ROCm][Perf] Fused MoE W4A16 HIP kernel for AMD RDNA3 (gfx1100) (#44075)
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-06-06 15:30:39 -05:00
Harry Mellor a80af24356 Speed up docs build (#44635)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-05 14:51:44 +00:00
Jee Jee LiandYiliu Dong 559d6710bf [PERF]MiniMax-M2 gate kernel (#38445)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: qianlihuang <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
2026-05-29 18:28:34 -07:00
JartX 0cff0741ff [Kernel][ROCm] Native W4A16 kernel for AMD RDNA3 (gfx1100) — fp16 + bf16 (#41394)
Signed-off-by: JartX <sagformas@epdcenter.es>
2026-05-29 11:04:40 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 710f077617 [Refactor] Remove dead code (#43234)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-29 00:29:56 -04:00
Jee Jee Li d4004455d2 [Kernel] Remove NormGateLinear (#43554)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-05-25 09:49:19 +00:00
Xiaochang WuandKunshang Ji 8c8b1825eb [XPU] Enable multiple key kernels for sparse attention (#37888)
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-22 12:02:51 +08:00
Wentao Ye b29cbf0652 [Perf] zeros -> empty to remove additional fill (#42988)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-21 16:00:29 -04:00
Wentao Ye 37ece593c1 [Perf] Padded nvfp4 quant kernel to remove additional copy, 2.4%~5.7% e2e performance improvement (#42774)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-18 16:38:12 -07:00
Wentao Ye 00e20e76f7 [Refactor] Remove dead cuda kernels (#42767)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-18 11:14:21 -07:00
Yuwen Zhou 88a860d754 [CPU] Add MXFP4 W4A16 MoE support (#41922)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
2026-05-18 03:04:45 -07:00
Li, Jiang b4601ad43f [CPU] Add fused GDN support for AMX CPU platform (#42707)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-05-18 03:04:36 -07:00
Jee Jee Lijeejeeleemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
0a65d46628 [DSV4] Fuse norm and router for low latency scenario (#41263)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io>
Co-authored-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-14 05:11:02 -07:00
Itay EtelisItay EtelisOr OzeriItay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
00b0618a03 Use CU_MEMCPY_SRC_ACCESS_ORDER_ANY for batch KV cache swaps (#39306)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <etelis2019@gmail.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <etelis2019@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-10 05:57:09 +03:00
Li, Jiang b3945cc316 [CPU] Bump up to the latest CPU kernels (#41924)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-05-07 05:45:59 -07:00
Yuwen Zhou 713b28bd0b [CPU] Add FP8 W8A16 MoE support (#41314)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
2026-05-06 23:17:07 -07:00
Yuwen Zhou 809b98e5b7 [CPU] Add FP8 W8A16 linear support (#41186)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
2026-05-06 07:05:27 +00:00
Woosuk Kwon 9c61864bf8 [DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm (#41300)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-04-30 16:28:57 -07:00
Tianmu LiClaudeLi, Jiang <jiang1.li@intel.com>
22524f7a92 [Feat] CPU fp8 attn for AMX/AVX-512 (#39445)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-04-29 20:43:21 +08:00
+7 4d51588e23 [Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
2026-04-26 18:31:08 -07:00
Wentao Ye 16688b26a6 [Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency improvement (#40413)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-04-21 19:51:03 +00:00
Michael Goin a8bffaa133 [Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-04-17 16:42:32 -07:00
Fadi Arafehandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 445b7093fd [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-15 22:26:17 -07:00
Kaicheng YangandClaude Sonnet 4.6 4beeb0689c fused qknorm+rope kernel optimization for SM9.0 (#37376)
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 19:58:37 -07:00
Jee Jee Li ecd1ea1363 [Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
2026-04-11 00:20:20 +08:00
4a06e1246e [Perf] Batch KV cache swap copies via cuMemcpyBatchAsync (#38460)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
2026-04-03 03:13:23 +00:00
Carl Y 3bc2734dd0 [Kernel] Fuse FP8 output quantization into merge_attn_states (#36518)
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
2026-04-03 01:47:04 +00:00