23 Commits
Author SHA1 Message Date
28e92eeff2 [ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill (#59552)
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-16 22:58:56 +00:00
Andreas KaratzasandClaude 67516b6c42 [Bugfix][ROCm][Kimi-K3] Stage fused KDA decode weights after loading (#61207)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-13 14:55:57 -05:00
Misha GoinandClaude d79b52b182 [Misc] Consolidate csrc shared headers and prune dead code (#60721)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-12 09:46:22 -07:00
Warren Dengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fc39317392 [Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys (#60446)
Signed-off-by: Warren Deng <warrdeng@meta.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-09 14:25:41 +00:00
Rohan PotdarandClaude Opus 5.5 88a419649e [ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization (#60717)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 04:45:13 +00:00
458ba2edf8 [ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization (#59069)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
2026-10-08 12:01:47 +00:00
kliuae 0154979656 [ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm (#58344)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-10-02 06:58:04 +00:00
Jaden Mathias d848c4ed3d [ROCm][Triton] Migrate Kimi-K3 kernels from make_block_ptr to tensor … (#58769)
Signed-off-by: JadenMathias <jaden.mathias@amd.com>
2026-10-01 13:41:41 -05:00
1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045)
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-25 12:33:30 -05:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Divakar Verma 06e57f622c [ROCm][Kimi-K3] Fix non-contiguous state_indices crash and GPU-sync assert in fused KDA/MLA prefill (#56526)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
2026-09-11 20:17:39 -05:00
9521c60bdc [ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill reland (#54038)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <17350011+kliuae@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-09-10 15:37:54 +08:00
Kevin H. LuuandAndreas Karatzas 592e06f2ae Revert "[ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill" (#53294)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-08-21 13:34:13 -07:00
463aa5e30f [ROCm][Perf] Kimi-K3 Fused kernels for KDA prefill (#52606)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-08-21 09:07:13 -07:00
Matvei PashkovskiiandClaude ad5e71b276 [ROCm][Perf] Enable fused KDA decode on gfx942 (MI325X) (#52293)
Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-18 18:01:00 +00:00
kliuae f96261637c [ROCm][Perf] Kimi-K3 Remove prefill pipeline stall in chunk KDA (#51862)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-08-13 09:11:45 +00:00
7f9173dfa2 [ROCm][Perf] Kimi-K3 Fused kernel for KDA decode (#50654)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
2026-08-11 20:39:34 -07:00
Lyu, Xudong 0e2d78028c [Bugfix][Kimi-K3] Give the AMD packed KDA decode kernel the state-index stride (#51682)
Signed-off-by: Lyu, Xudong <xudong.lyu@amd.com>
2026-08-11 00:05:20 +08:00
kliuae f5cd862dbd [ROCm][Bugfix] Kimi-K3 Fix KDA NaN on mixed batches and racy autotune config (#50649)
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
2026-08-05 08:07:21 +00:00
yinfengLiuLiuyinfeng01mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7ac2ec7582 [Kimi-K3][AMD] Fuse AttnRes state updates and norms (#50593)
Signed-off-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 12:32:13 -04:00
+12 7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
2026-07-29 14:10:58 +08:00
Harry Mellor 62d8db7c05 [Bugfix] Add missing vllm/models/kimi_k3/__init__.py (#50131)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-07-28 13:47:04 +00:00
Thien Tran 61ac368021 [Kimi-K3] Add AttnRes kernels (#50090)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-07-28 00:22:09 -07:00