[MM Encoder]: Make MMEncoderAttention's scale takes effect properly (#31950)

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
Isotr0py
2026-01-08 02:33:48 -08:00
committed by GitHub
parent 5576227bc1
commit 2972a05473
11 changed files with 32 additions and 8 deletions
+1
View File
@@ -188,6 +188,7 @@ class GlmAsrEncoderAttention(nn.Module):
self.attn = MMEncoderAttention(
num_heads=self.num_heads_per_rank,
head_size=self.head_dim,
scale=self.head_dim**-0.5,
num_kv_heads=self.num_kv_heads_per_rank,
prefix=f"{prefix}.attn",
)