[Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061)

Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Jared Wen
2026-09-22 21:58:21 +08:00
committed by GitHub
co-authored by Claude Fable 5.1 mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent d90f0eade5
commit c9b34fdb2d
2 changed files with 115 additions and 0 deletions
+1
View File
@@ -355,6 +355,7 @@ class Glm5NextDecoderLayer(nn.Module):
intermediate_size=config.intermediate_size,
hidden_act=config.hidden_act,
quant_config=quant_config,
is_sequence_parallel=self.is_sequence_parallel,
prefix=f"{prefix}.mlp",
swiglu_limit=config.swiglu_limit,
)