[ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 (#39999)

Signed-off-by: Hemanth Acharya <heachary@amd.com>
This commit is contained in:
Hemanth Acharya
2026-04-24 09:02:12 +09:00
committed by GitHub
parent 447c372ac5
commit fa4b70555b
4 changed files with 19 additions and 2 deletions
+2
View File
@@ -1782,6 +1782,8 @@ class rocm_aiter_ops:
need_renorm: bool,
routed_scaling_factor: float = 1.0,
) -> None:
if correction_bias.dtype != gating_output.dtype:
correction_bias = correction_bias.to(gating_output.dtype)
torch.ops.vllm.rocm_aiter_biased_grouped_topk(
gating_output,
correction_bias,