[Kimi-K3][Perf] Make native CUDA AttnRes the SM100 default (#54261)

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
This commit is contained in:
Yongye Zhu
2026-08-31 23:18:12 +00:00
committed by GitHub
parent 07ea9350ba
commit d6d6658543
7 changed files with 415 additions and 94 deletions
+5 -3
View File
@@ -542,9 +542,11 @@ STABLE_TORCH_LIBRARY_FRAGMENT(_C, ops) {
#ifdef VLLM_ENABLE_KIMI_K3_ATTN_RES
ops.def(
"kimi_k3_attn_res("
"Tensor! prefix, Tensor delta, Tensor blocks, Tensor norm_weight, "
"Tensor qk_weight, Tensor output_norm_weight, Tensor! output, "
"int num_blocks, float eps, float output_norm_eps) -> ()");
"Tensor! prefix, Tensor? delta, Tensor! blocks, Tensor norm_weight, "
"Tensor qk_weight, Tensor? output_norm_weight, Tensor! output, "
"int num_blocks, "
"int block_write_idx, float eps, "
"float output_norm_eps) -> ()");
#endif
// Apply repetition penalties to logits in-place.