[ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation (#51473)

Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
This commit is contained in:
Fangzhou Ai
2026-08-10 23:57:27 -07:00
committed by GitHub
parent be2e274ef5
commit 1044d19998
2 changed files with 63 additions and 3 deletions
@@ -651,6 +651,12 @@ def mxfp4_round_up_hidden_size_and_intermediate_size(
# which would inflate weights and OOM.
intermediate_size = round_up(intermediate_size, 128)
hidden_size = round_up(hidden_size, 128)
elif (
backend == Mxfp4MoeBackend.AITER_MXFP4_BF16 and activation == MoEActivation.SILU
):
# AITER's A16W4 SiLU kernel handles native dimensions aligned to 128.
intermediate_size = round_up(intermediate_size, 128)
hidden_size = round_up(hidden_size, 128)
elif backend == Mxfp4MoeBackend.EMULATION:
# Emulation has no kernel tile; it only needs OCP MX block alignment so the
# per-block scale buffers (`dim // OCP_MX_BLOCK_SIZE`) aren't floor-truncated