[BugFix] Accuracy fix for llama4 int4 - improperly casted scales (#16801)

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
This commit is contained in:
Lucas Wilkinson
2025-04-17 22:13:29 -07:00
committed by GitHub
parent 6a0f547561
commit 7eb4255628
3 changed files with 6 additions and 9 deletions
@@ -422,6 +422,7 @@ class FusedMoE(torch.nn.Module):
if params_dtype is None:
params_dtype = torch.get_default_dtype()
self.params_dtype = params_dtype
# Note: here we guard against accessing the TP and DP groups when
# uninitialized (this happens when testing)