 Richard ZouandCodex
|
c27b080d26
|
[compile] Fix fake kernel return dtype (#50444)
Signed-off-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-07-30 19:50:02 +00:00 |
|
|
+13
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
|
yinfengLiu
|
f1e8fd27ae
|
[ROCm] Add AITER FP8 ViT encoder attention (#49937)
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
|
2026-07-30 06:22:10 +00:00 |
|
|
Yiliu DongandOpenAI Codex
|
ba702e978e
|
[Attention] Skip sparse indexer scoring for dense short prefills (#48407)
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 16:17:22 +00:00 |
|
|
|
28158b2fc3
|
[ROCm] [BugFix] Fix Quark GLM-5.2 Checkpoint inference: indexer wk per-channel FP8 dequant + missing sparse-MLA metadata fields (#48886)
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Signed-off-by: ColinZ22 <Colin.Zeng@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: fanxingran <xingran.fan@amd.com>
|
2026-07-27 22:38:22 +00:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
060b5f61dc
|
[Bugfix][Attention] Ignore empty MLA context chunks during merge (#49294)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-22 02:23:48 +00:00 |
|
|
|
2396a61108
|
[Attention][MLA][DCP] Query replication for MLA decode (DeepSeek-V2/R1 + Kimi-K2.5) (#45964)
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-20 23:51:27 +00:00 |
|
|
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
|
aoshen02
|
17fdd42100
|
[Bugfix][Attention] Preserve post-load tensors across weight reloads (#48251)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-07-17 14:15:26 +08:00 |
|
|
|
67f9046e4a
|
[Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-16 22:44:04 +00:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f17be06fbe
|
[Perf] Optimize clamp to clamp_ (#48143)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-16 18:41:07 -04:00 |
|
|
Jack HuandDouglas Lehr
|
2cab53ddee
|
[Model][Hardware][AMD]: Part 1/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_AITER_FA, and ROCM_AITER_UNIFIED_ATTN (#42749)
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-07-16 17:39:04 -05:00 |
|
|
|
0becb7486b
|
[BugFix][MLA] Support kv_cache_dtype_skip_layers for MLA attention (#47309)
Signed-off-by: liuruikang <liuruikang.cs@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-16 00:06:11 +00:00 |
|
|
drakoshaandMatthew Bonanni
|
6472131298
|
[Bugfix] Set kv_quant_mode on the generic MLA KV-cache spec (#48379)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-15 03:36:52 +00:00 |
|
|
Michael Goin
|
b2f7d2560a
|
[Bugfix] Make MLA+SWA check the layer's backend, not the model config (#48520)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-14 09:53:34 -07:00 |
|
|
Matthew BonanniandClaude Opus 4.8
|
c4f5cd60da
|
[1/N] Add dense MHA path for sparse MLA short sequences (#47327)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-14 00:29:56 +00:00 |
|
|
|
bea70c7cfc
|
[Attention] Make sliding-window support an explicit backend capability (#48011)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-13 01:07:56 -07:00 |
|
|
Martin VitandMatthew Bonanni
|
c2ecd0f888
|
Fix FlashAttention MLA prefill V unpadding (#42642)
Signed-off-by: Martin Vit <martin@voipmonitor.org>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-08 15:22:20 +00:00 |
|
|
sychen52andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
67ff0ae30f
|
Support nvfp4 kv with kv-cache-dtype-skip-layers sliding_window (#42890)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-04 02:29:13 +00:00 |
|
|
Luciano MartinsandLuciano Martins
|
979f5511d7
|
[Bugfix][Gemma4] Keep image bidirectional attention within the sliding window (#47217)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-07-02 19:57:41 -07:00 |
|
|
Andreas Karatzas
|
c8f9c156a5
|
[ROCm][V1][MLA] Clone prefill backend state per metadata builder (#46993)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-30 11:43:54 -07:00 |
|
|
|
8cf7c4d8ad
|
[Attention Backend] add HPC-Ops Attention backend (#46020)
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-30 18:17:43 +08:00 |
|
|
Summer YangandClaude
|
5b4cb69523
|
[Bugfix][MLA] Fix LSE log-base mismatch in DCP + FlashInfer MLA decode (#47079)
Signed-off-by: girasoley <girasoleyang@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-29 19:15:02 -07:00 |
|
|
|
c6741b2ad4
|
[Model] Support Unlimited OCR (#46564)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-06-27 23:09:18 -07:00 |
|
|
|
710ebaa189
|
[ROCm][Bugfix] Fix chunk alignment when using context parallelism with TRITON_MLA (#46114)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-25 00:07:28 -04:00 |
|
|
|
49f2104c53
|
[Feature] Support DCP with FP8 KV cache in MLA decode path (#44044)
Signed-off-by: shivampr <shivampr.dev@gmail.com>
Signed-off-by: Shivam <shivamprasad91@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-24 19:28:17 +00:00 |
|
|
|
44d95069e9
|
Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-06-22 11:54:14 -07:00 |
|
|
Yifan Qiao
|
aa4990a9a2
|
[Attention] Re-enable cross-layer KV cache layout for MLA via stride-aware kernels (#45111)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-22 06:57:02 -07:00 |
|
|
Athrael SojuandClaude Opus 4.8
|
3c8e49596c
|
[Model] ColQwen3.5: fix retrieval correctness (bias + bidirectional) (#46108)
Signed-off-by: Athrael Soju <athrael.soju@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-22 17:25:54 +08:00 |
|
|
Nick Hill
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
|
Jared Wen
|
ab66606993
|
[bugfix]Indexer init skip and MTP TopK share for iteration (#45895)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
|
2026-06-19 09:57:51 +08:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
79ca54d221
|
[Bugfix][Quantization] Don't reject fp8_e5m2 KV cache for non-fp8 quantized checkpoints (#45040)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-18 14:18:25 -04:00 |
|
|
yifei wuandHarry Mellor
|
a52205bccf
|
[Model] Add HrmTextForCausalLM (Hierarchical Reasoning Model — Text) (#43098)
Signed-off-by: Wuyifei <wuyifei@me.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-16 22:41:41 +08:00 |
|
|
Carl Y
|
eb04c769d3
|
feat: MLA prefill enable FA4 fp8 output (#43050)
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
|
2026-06-16 07:10:59 -07:00 |
|
|
|
0a1c5034f5
|
[Model] Add MiniMax M3 support (#45381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-16 01:01:25 +08:00 |
|
|
wentian-byte
|
b8142294b7
|
[Bugfix] Restrict FlashInfer cuDNN FP8 ViT attention gate to Blackwell (SM 100) (#45251)
Signed-off-by: Wentian Byte <3400259131@qq.com>
|
2026-06-11 16:39:24 +00:00 |
|
|
wineandchord
|
c2b4cd39ac
|
[Doc][Attention] Fix MLA top-of-file comments (#37047)
Signed-off-by: wineandchord <guoqizhou19@gmail.com>
|
2026-06-11 08:14:45 -07:00 |
|
|
qizixi
|
e2db0222e9
|
[Perf][Attention] Pin MLA chunked-context metadata tensors so H2D copies are truly non-blocking (#45074)
Signed-off-by: zixi-qi <zixi@inferact.ai>
|
2026-06-10 15:56:49 -07:00 |
|
|
nightcitybladeandnightcityblade
|
8b8546da1c
|
docs: fix MLA attention docstring examples (#44118)
Co-authored-by: nightcityblade <nightcityblade@gmail.com>
|
2026-05-31 12:28:38 -07:00 |
|
|
Jiangyun Zhu
|
8a56da3845
|
[Experimental] Breakable CUDA graph (#42304)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-05-16 22:04:12 +08:00 |
|
|
Michael Goin
|
852f567444
|
[Bugfix] Respect explicit --kv-cache-dtype over checkpoint kv_cache_scheme (#42782)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-05-15 17:15:52 -07:00 |
|
|
Wentao Ye
|
491e8d8539
|
[Perf] Optimize MLA attention _v_up_proj bmm by removing additional copy (#42561)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-15 08:14:26 -07:00 |
|
|
Matthew Bonanni
|
9898f94abe
|
[Attention] Remove deprecated MLA prefill arguments (#42555)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-14 10:34:06 -07:00 |
|
|
Yongye ZhuandRoger Wang
|
0d2732dd91
|
[MLA Attention Backend] Add TOKENSPEED_MLA backend for DSR1/Kimi K25 prefill + decode on Blackwell (#41778)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-05-13 23:48:02 -07:00 |
|
|
liangel-02
|
6b5c389ee3
|
expose flex block size for batch invariant mode (#41252)
Signed-off-by: Angel Li <liangel@meta.com>
|
2026-05-13 14:11:57 -07:00 |
|
|
Wentao Ye
|
3d635c58c0
|
[Perf] Optimize MLA compute_prefill_context memory allocation (#42460)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-12 16:23:46 -07:00 |
|
|
Rohan PotdarandElizaWszola
|
a51376b3f0
|
[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
|
2026-05-11 14:10:50 +00:00 |
|
|
Matthew Bonanni
|
01b9b5af67
|
[Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-05 23:22:41 +00:00 |
|
|
Andreas Karatzas
|
1cb0838721
|
[ROCm][CI] Fix MLA prefill scale for DeepSeek GSM8K (#41569)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-05-04 16:32:55 -07:00 |
|
|
|
f3fef12350
|
[Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-01 13:36:20 -04:00 |
|