[Bugfix] Store Qwen3Next A_log in fp32 (#37810)

Signed-off-by: effortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
Hojin Yang
2026-03-23 15:36:57 +08:00
committed by GitHub
co-authored by Roger Wang
parent 02e6efe56d
commit 54ab804e87
+1
View File
@@ -501,6 +501,7 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase):
self.A_log = nn.Parameter(
torch.empty(
divide(self.num_v_heads, self.tp_size),
dtype=torch.float32,
)
)