[vLLM IR] rework gemma_rms_norm (#39014)

Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
This commit is contained in:
Jiangyun Zhu
2026-04-07 01:37:00 -07:00
committed by GitHub
co-authored by Luka Govedič
parent da4c0e4db9
commit 8060bb0333
8 changed files with 106 additions and 75 deletions
+2 -3
View File
@@ -16,7 +16,6 @@ def rms_norm(
x_var = x if variance_size is None else x[..., :variance_size]
variance = x_var.pow(2).mean(dim=-1, keepdim=True)
x = x * torch.rsqrt(variance + epsilon)
x = x.to(orig_dtype)
if weight is not None:
x = x * weight
return x
x = x.to(weight.dtype) * weight
return x.to(orig_dtype)