[Kimi-K3] Overlap low-M TP8 KDA projections (#54697)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
12b9573c98
commit
3ba9907a1d
@@ -895,6 +895,7 @@ class KimiDecoderLayer(nn.Module):
|
||||
config,
|
||||
vllm_config,
|
||||
prefix=f"{prefix}.self_attn",
|
||||
aux_stream=aux_stream,
|
||||
run_gemm_rs_ar=run_gemm_rs_ar,
|
||||
)
|
||||
self._self_attn_writes_output = False
|
||||
|
||||
Reference in New Issue
Block a user