 Wentao Ye
|
c19c263ac8
|
[GLM 5.3 Perf] Handle two elements at the same time for per token group quant, 2.8% E2E ITL improvement (#60927)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-13 13:06:41 -07:00 |
|
|
|
e83222396f
|
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199)
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-10-13 17:21:30 +00:00 |
|
|
Elvir Crnčević
|
b9b401b2ef
|
[Kernel] Fuse DeepEP-v2 decode globalize+align into one launch (#53911)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
|
2026-10-13 11:10:38 -04:00 |
|
|
Thien Tran
|
0fae39a014
|
[Perf] per_token_group_quant_8bit: multiply by reciprocal instead of dividing per element (#61343)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-10-13 21:37:10 +08:00 |
|
|
Yongye ZhuandClaude Opus 5.5
|
a6009ea85b
|
[Kernel][Build] Bump DeepGEMM to the 26/09/30 release and adapt the sparse indexer (#61149)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-12 23:24:43 -07:00 |
|
|
Misha GoinandClaude
|
d79b52b182
|
[Misc] Consolidate csrc shared headers and prune dead code (#60721)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 09:46:22 -07:00 |
|
|
Wentao Ye
|
838526c7e5
|
[Compile] Fix sampler compile warning (#60680)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-10 08:09:45 -07:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
d5f356dae0
|
[HiSparse] Own GPU residency instead of editing core block-table rows (#60647)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 21:25:44 -07:00 |
|
|
|
a6f43e57d6
|
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
|
2026-10-09 08:29:55 +00:00 |
|
|
Rehan KhanandBob
|
fd0216d3b5
|
[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias (#60221)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Bob <bob@ibm.com>
|
2026-10-08 15:15:53 +08:00 |
|
|
MattandClaude Opus 5.5
|
5dd4a628dd
|
[ROCm][Bugfix] Map sleep-mode cuMem memory for the host (#60367)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-08 04:26:12 +00:00 |
|
|
Matthias Gehre
|
b267fe4881
|
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides (#56301)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-10-06 08:22:33 -05:00 |
|
|
Mia Ojeda
|
3313825983
|
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step (#59992)
Signed-off-by: ojeda-e <eojeda@league.com>
|
2026-10-06 19:25:51 +08:00 |
|
|
|
4ac0d0eac2
|
[ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism (#54706)
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
|
2026-10-03 23:44:17 +08:00 |
|
|
|
6660926d92
|
[Bugfix][CPU] Fix macOS build on Apple Clang < 17: structured binding… (#58932)
Signed-off-by: Miajan Chowdhury <chowdhurymiajan@gmail.com>
Co-authored-by: Miajan Chowdhury <chowdhurymiajan@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-02 10:01:07 +08:00 |
|
|
Yongye ZhuandClaude Opus 5.5
|
7566d83bd3
|
[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path (#59300)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-01 14:39:55 -07:00 |
|
|
almayneandLi, Jiang <jiang1.li@intel.com>
|
93dd80f636
|
[CPU] Conv1d optimised kernel for aarch64 (#54093)
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-30 20:23:42 +08:00 |
|
|
Rukhaiya2004andLi, Jiang <jiang1.li@intel.com>
|
72e7874fa6
|
[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX (#59149)
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-30 19:44:48 +08:00 |
|
|
Rehan KhanandLi, Jiang <jiang1.li@intel.com>
|
583637f126
|
[CPU][Perf] Add vectorized Sampler Kernel (#53913)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-30 15:16:28 +08:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
2798f66860
|
[Bugfix][HiSparse] Resolve MTP verification rows with a union residency kernel (#59235)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-30 03:01:45 +00:00 |
|
|
Yongye ZhuandClaude Opus 5.5
|
77871126f9
|
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-27 03:08:36 +08:00 |
|
|
|
38cc054fc2
|
[Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194)
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-09-25 09:19:48 -06:00 |
|
|
chuan932andMisha Goin
|
f550ad8bf7
|
[Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330)
Signed-off-by: chao.huan <chao.huan@nio.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-09-25 08:11:42 -07:00 |
|
|
Yongye ZhuandSummer Yang
|
442bd00d9f
|
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
|
2026-09-25 03:27:21 +00:00 |
|
|
|
f34a0e0797
|
[CPU] Gate the AVX10.2 paths on compiler support (#58133)
Signed-off-by: R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-24 18:07:53 +08:00 |
|
|
Ankit-Jaiswal-AMDandDouglas Lehr
|
88afb9dcac
|
[CPU] Adds support for fp32 attention sinks (#56252)
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-09-23 08:55:09 -05:00 |
|
|
Gabriel WuandKimi Code
|
c843f0cab7
|
[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend (#55277)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
|
2026-09-23 02:03:20 -07:00 |
|
|
Misha Goin
|
25965ee583
|
[Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-22 14:07:34 -07:00 |
|
|
Gilles TurpinandWentao Ye
|
74370a0e30
|
[Bugfix] hadacore_transform: respect inplace parameter to fix garbage outputs with QuIP transforms (#43462)
Signed-off-by: Gilles Turpin <turpingilles15@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-09-22 12:48:20 -04:00 |
|
|
Woosuk KwonandCodex
|
9a70c233cd
|
[Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-20 20:28:06 -07:00 |
|
|
Wentao Ye
|
4868312128
|
[Refactor] Remove dead kernel code (#57621)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-20 10:39:23 -07:00 |
|
|
|
7d99c2c4fd
|
[Feature][Humming] Humming feature integration (#56685)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-20 08:11:01 -07:00 |
|
|
|
92b40f5a14
|
[CPU][Perf] refactor paged attention for Arm CPUs (#56045)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-20 12:25:13 +08:00 |
|
|
|
a7fda4c88b
|
[Bugfix] Fix SM100 fp8_ds_mla cache scales (#49435)
Signed-off-by: Ariel Bereslavsky <arielb@getimpala.ai>
Signed-off-by: Ariel Bereslavsky <ariel@getimpala.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ariel Bereslavsky <arielb@getimpala.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-19 17:15:07 -07:00 |
|
|
Yuwen ZhouandLi, Jiang <jiang1.li@intel.com>
|
b346479065
|
[CPU] Add CPU FP8 W8A8 linear/MoE support (#49942)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-18 17:59:42 +08:00 |
|
|
Thien TranandCodex
|
8da75d6190
|
C3x SM100 FP8 blockwise - Pad activation scales to multiple of 4 (#57377)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-18 08:39:27 +00:00 |
|
|
|
117cf43fb6
|
[HARDWARE][POWER] Enable W8A8 INT8 MoE on POWER (#55316)
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-09-17 21:45:13 +08:00 |
|
|
Wentao Ye
|
91a4c40b45
|
[Compile] Fix compile warning #177-D (#56609)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-17 05:04:56 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
|
d6a1677d55
|
[Model][DSv4.1] FlashMLA mega attention and the NVFP4 compressed KV cache (#56935)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Yunxiao Ning <73625538+foraxe@users.noreply.github.com>
Co-authored-by: AI Assistant <noreply@openai.com>
|
2026-09-16 03:07:50 -07:00 |
|
|
|
6ecd97f1b7
|
[Perf][DSpark] Add KV-only context insertion across V4.1 cache formats (#56441)
Signed-off-by: 0z5a <0z5a@users.noreply.github.com>
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: 0z5a <0z5a@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 21:34:55 -07:00 |
|
|
Misha GoinandOpenAI Codex
|
c6fa1f05d1
|
[Perf][Kernel] Add sampled filtering for persistent top-k (#56346)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-15 14:15:59 -07:00 |
|
|
Fangzhou Ai
|
dffbb714e4
|
[ROCm][Perf] Optimize DSV4.1 K=512 decode top-k on gfx950 (#56743)
Signed-off-by: fai <fangzhouai@gmail.com>
|
2026-09-15 18:34:24 +00:00 |
|
|
|
e6eb0d120c
|
[Model][DSv4.1] Store the whole KV in MXFP8 (FlashMLA V4.1 record) (#56893)
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-14 20:12:55 -07:00 |
|
|
sychen52
|
7702ee87db
|
[Bugfix][DSA] Write nvfp4_ds_mla from the fused norm+rope kernel (#55538)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-09-15 00:25:33 +04:00 |
|
|
Matthias Gehre
|
dc2e8f1157
|
[ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit (#54965)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-09-14 17:40:25 +08:00 |
|
|
Ayushman SinghandAyushman Singh
|
c2ea9f1a5f
|
[Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization (#56478)
Signed-off-by: Ayushman Singh <ayush1399@users.noreply.github.com>
Co-authored-by: Ayushman Singh <ayush1399@users.noreply.github.com>
|
2026-09-12 20:27:08 -07:00 |
|
|
|
29332cf936
|
[4/N] Expose HiSparse cache metrics via KV connector stats (#56061)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 10:47:55 -07:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
Misha Goin
|
8a7f98c8c3
|
[Kernel][MoE] Optimize batched_moe_align_block_size with cooperative writes (#53280)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-11 12:05:22 -07:00 |
|