Jiangyun Zhu
98ed0856f3
[Model] add GLM-5.3-Flash support ( #53906 )
2026-09-03 09:40:35 -07:00
Isotr0py
1356635d83
[New model][Multimodal] Add DeepSeek-V4-Flash-Vision-Exp support ( #54566 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-02 11:09:09 -07:00
Luke Alonso and Lucas Wilkinson
0d4ad47981
[Kernel] Add B12X causal paged attention backend ( #52017 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-01 10:42:02 -04:00
Andrii Skliar and Andrii Skliar
adebc41b7e
[Mamba] Add FlashInfer ReplaySSM backend ( #52506 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-01 10:26:43 -04:00
sychen52
7c5dc571cb
[Attention][DSA] Enable W4A16 DSA ( #51724 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-09-01 12:37:47 +08:00
Roberto L. Castro and Codex
e16b5e518d
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels ( #50175 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-31 23:26:05 -04:00
dbb7fffddb
[ROCm][MLA][DCP] Support causal multi-token verification ( #51705 )
...
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com >
Signed-off-by: billishyahao <bill.he@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: billishyahao <bill.he@amd.com >
2026-08-31 10:42:28 -07:00
andyluo7 and OpenAI Codex
76ff0cdff2
[Bugfix][ROCm] Preserve AITER unified-attention metadata during graph replay ( #53821 )
...
Signed-off-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-31 21:53:12 +08:00
Li, Jiang
fdbf2ddbd2
[Bugfix][CPU] Fix several bugs ( #54042 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-08-31 15:33:06 +08:00
Lucas Wilkinson
5707355209
[Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA ( #54465 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-31 01:13:42 +00:00
dbf662c9e8
[ROCm][MLA] Reach FULL cudagraphs for AITER MLA speculative decoding ( #51171 )
...
Signed-off-by: yudigege86 <naqin@amd.com >
Co-authored-by: yudigege86 <naqin@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-29 19:49:55 -07:00
Summer Yang
6d4562c59b
[Attention][DCP] Enable FlashInfer MLA for DSpark drafting ( #54277 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-28 21:11:28 -07:00
maobaolong and Codex
6f7df92a8e
[CPU][MLA] Fix prefill backend selection so MLA runs end-to-end on CPU ( #51471 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-28 12:36:55 +08:00
Matthew Bonanni and OpenAI Codex
de9250ac9e
[Attention] Enable masked MHA for GLM-5 head dimensions ( #53785 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-27 21:17:12 +00:00
Summer Yang
07242faa46
[Attention][DCP] Use FlashInfer native CP for MLA decode ( #54012 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-27 10:32:45 -07:00
Chauncey
d21c5b50a1
[Perf][GLM5.2] Fuse sparse MLA Q concatenation with head padding ( #53878 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-27 11:54:32 -04:00
Misha Goin
7de96050c5
[Spec Decode] Enable adaptive DSpark on SM100 sparse MLA ( #52783 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-25 13:31:15 -04:00
Misha Goin
f620499ee3
[Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata ( #53336 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-24 08:08:43 -07:00
Andreas Karatzas
30b34171b1
[ROCm][CI] Restore attention coverage after KV-cache layout refactor ( #53351 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-22 20:00:25 -05:00
bbe8b23e1a
[Deprecation] Remove dead use_prefill_decode_attention flag ( #52557 )
...
Signed-off-by: Brian Woods <me@brian.biz >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 22:38:10 +00:00
Lucas Wilkinson
8bdc70ec7b
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout ( #51718 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-21 17:47:49 -07:00
Misha Goin
9ff7041b55
[Bugfix][Spec Decode] Use group geometry for FlashAttention metadata ( #53002 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-21 17:07:19 -07:00
Summer Yang
d6c2fec9fd
[Cleanup][MLA] Remove FlashInfer DSpark DCP support ( #53139 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-21 11:42:52 -07:00
yimdev and usberkeley
574e6a00ef
[Bugfix][Attention] Normalize FlashInfer prefill LSE before merging ( #52796 )
...
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com >
Co-authored-by: usberkeley <150880684+usberkeley@users.noreply.github.com >
2026-08-21 11:31:14 -04:00
Eoin-Houstoun and Claude Fable 5
30e2394c83
[Bugfix] Record non-ImportError attention backend probe failures instead of crashing engine init ( #51703 )
...
Signed-off-by: Eoin <eoin@turintech.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-20 02:36:00 -07:00
Summer Yang and Woosuk Kwon
58e5ee0158
[refactor] consolidate cp attn ops ( #52839 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-19 18:04:11 -07:00
63ff748f65
[Attention][MLA] FlashMLA sparse: DCP on the fp8_ds_mla mixed-batch path + MTP ( #46514 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-19 00:02:09 -04:00
Matthew Bonanni and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8e46accab2
[Attention] Vectorize sparse MLA mask loads ( #52217 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 20:32:41 -07:00
Li, Jiang
e4d61d0d22
[CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 ( #52616 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-08-19 10:28:52 +08:00
yinfengLiu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
5d8a4cf976
[ROCm] Pad non-aligned AITER MLA heads ( #51647 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 16:37:10 -05:00
0db502c8d8
[Kimi-K3] support DCP partial prefix cache hit ( #50493 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-08-17 17:05:51 -07:00
Tommy Asai and Roger Wang
58aa1e3d26
[Bugfix][SM120][MLA] Disable dense prefill for FlashInfer sparse MLA ( #51395 )
...
Signed-off-by: tommy-asai-sonarsource <yosuke.asai@sonarsource.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-17 16:58:02 -07:00
Wei Zhao
d1e3eee6fb
[Spec decode] Support Kimi-K3 DCP with DSpark ( #52188 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-08-17 13:08:45 -07:00
f27ae25473
[Bugfix][CPU] Take an attention group's query head count from its layers ( #51852 )
...
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Signed-off-by: R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-17 18:45:59 +08:00
Gabriel Wu
97388c44f9
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark ( #51538 )
2026-08-15 07:01:32 -07:00
Zhuobin Huang and nodeeeeee
63a9a5010a
[Attention][DSA] Take the native decode path for MTP=3 on SM90 ( #52164 )
...
Signed-off-by: zobinHuang <zobin1999@gmail.com >
Co-authored-by: nodeeeeee <zhangkai.nodeee@gmail.com >
2026-08-14 10:09:51 +00:00
b652dedd0c
[Attention] Fix FlashInfer SM12x prefill with sinks ( #52148 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-08-13 14:25:09 -07:00
903d2efe7e
[Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing ( #51772 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 22:42:57 -07:00
9035151d6c
[Model] Add native Dots3 NOTE multimodal support ( #51255 )
...
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-12 23:13:20 +08:00
1d2d83a07f
[Attention] Add FlashInfer XQA decode support on SM12x ( #49718 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 16:32:49 -07:00
Nick Hill
513f83e7ec
[Bugfix] Take the sliding window from the layer, not the KV cache group ( #51756 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-11 09:10:46 -07:00
63ac04a61e
[Kimi-K3] DCP support ( #50484 )
...
Signed-off-by: girasoley <girasoley@inferact.ai >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com >
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-10 12:54:26 -07:00
fanxingran
37fbf52084
[ROCm][MLA] [K3] Fix fp8 KV cache decode on the AITER MLA backend ( #51011 )
...
Signed-off-by: fanxingran <xingran.fan@amd.com >
2026-08-10 11:42:10 -04:00
Nick Hill and Claude Opus 5
9e6be4a72b
[Perf][Sparse MLA] Drop the atomic contention in the index remap ( #50365 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-08 16:32:40 -07:00
b38e111d3e
[Attention][MLA] Per-request scheduling for MLA chunked context ( #50613 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 16:45:44 +00:00
maobaolong and Li, Jiang <jiang1.li@intel.com>
13726c80fe
[CPU] Add MLA backend so DeepSeek-V2/V3 can run on CPU ( #49453 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-08-06 16:51:41 +08:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
NVShreyas and OpenAI Codex
8ae8337ffa
[Spec Decode] Enable fused non-causal TokenSpeed MLA for DSpark ( #50911 )
...
Signed-off-by: Shreyas Misra <shreyasm@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-04 18:05:40 +00:00
199644d410
[Bugfix][Attention] Guard sparse MLA masked MHA workspace ( #50906 )
...
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-08-04 09:59:33 -04:00
Matthew Bonanni
a0cd2b69b3
[Bugfix] Universally align block table width to 128 tokens ( #50302 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-31 08:52:17 -07:00
Jiangyun Zhu