[Kimi-K3] Overlap low-M TP8 KDA projections (#54697)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Yongye Zhu
2026-09-02 03:35:28 +00:00
committed by GitHub
co-authored by OpenAI Codex
parent 12b9573c98
commit 3ba9907a1d
7 changed files with 1092 additions and 12 deletions
+1
View File
@@ -895,6 +895,7 @@ class KimiDecoderLayer(nn.Module):
config,
vllm_config,
prefix=f"{prefix}.self_attn",
aux_stream=aux_stream,
run_gemm_rs_ar=run_gemm_rs_ar,
)
self._self_attn_writes_output = False