MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
dingdangmao and Codex
124b89d5b7
[Bugfix][RecoverSSM] Correct aligned state indices at exact block boundaries ( #59962 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-12 02:08:14 +00:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
766b4ae54c
[Kimi-K3] Add FlashInfer speculative KDA backend ( #54255 )
...
Signed-off-by: Duncan Moss <dmoss@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-01 02:28:50 +00:00
Jürgen Schmied and Claude
a80e41d690
[Mamba] Share RecoverSSM commit kernels and helpers across models
...
RecoverSSM's commit-plan and conv-window compaction kernels are model independent.
Move them from models/kimi_k3 into model_executor/layers/mamba/ops/recoverssm.py
(Kimi-K3 keeps its private names as import aliases; kernel source unchanged), add
mamba/recoverssm_utils.py (activation predicate, argument validation) and the
generic spec-row/request mapping helpers to v1/attention/backends/recoverssm_metadata.py.
CacheConfig.use_kda_recoverssm becomes use_recoverssm: the switch now selects RecoverSSM
for any model whose recurrent layers implement it, not only KDA.
No functional change for Kimi-K3.
Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-27 00:07:04 +02:00
1417022c3d
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA ( #58045 )
...
Signed-off-by: jiacao-amd <jiahui.cao@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:33:30 -05:00
Chauncey and zjy0516
3df4ae153e
[Core][KDA] Generalize Mamba prefill checkpoint builder and exporter ( #57783 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-21 04:46:11 +00:00
Wei Zhao
9d88ceb026
[KDA] Update flashKDA to support bf16 checkpoint state ( #56485 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-11 23:53:24 +00:00
Wentao Ye
86aca66191
[Kimi K3 Perf] Avoid KDA mixed-batch gather/scatter, 5.2%~7.7% E2E Throughput Improvement ( #56159 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-10 17:57:01 +08:00
Wei Zhao
a69402aaa8
[Perf] Integrate FlashInfer KDA kernels ( #55364 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-07 20:29:30 -07:00
Thien Tran
f4e6136146
[Kimi-K3] Bump FlashKDA to fix unstable inverse ( #54859 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-02 00:16:29 -07:00
Canlin Guo
aa640684cc
[Kimi K3][Kernel] Support DS conv-state layout in fused KDA decode kernel ( #53396 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
2026-08-26 23:54:50 -07:00
Kevin H. Luu and Codex
4c56e62c85
[Bugfix][Kimi K3] Skip absent metadata during CUDA graph profiling ( #53581 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-24 15:16:28 +00:00
9eb9d9d395
[Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement ( #52789 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-22 10:32:40 -07:00
70afdedc10
[K3] support recoverssm for K3 ( #51855 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-17 19:34:14 +08:00
pmanczak
cc7cf71fc8
[XPU] Enable Kimi K3 KDA kernel tests on XPU ( #51809 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
2026-08-17 17:21:34 +08:00
Lyu, Xudong
0e2d78028c
[Bugfix][Kimi-K3] Give the AMD packed KDA decode kernel the state-index stride ( #51682 )
...
Signed-off-by: Lyu, Xudong <xudong.lyu@amd.com >
2026-08-11 00:05:20 +08:00
+12
7c6729b769
[Model] Add Kimi K3 support: model files and kernels [1/N] ( #50089 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-07-29 14:10:58 +08:00