[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout (#60388)

Signed-off-by: JartX <sagformas@epdcenter.es>
This commit is contained in:
JartX
2026-10-08 15:06:39 +00:00
committed by GitHub
parent d80ddaac73
commit 994b8b77be
3 changed files with 95 additions and 9 deletions
@@ -926,15 +926,18 @@ def _process_weights_rdna3(
]:
"""RDNA3 (gfx1100) W4A16 weight post-processing.
Interleaves the packed nibbles per expert (the exllama shuffle the dense
RDNA3 kernel also uses) and synthesizes the symmetric zero points that
``moe_gptq_gemm_rdna3`` dequantizes with. The packed layout
``[E, K // 8, N]`` and the ``[E, groups, N]`` scales are already what the
kernel wants, so neither is repacked.
Transposes the canonical N-first inputs to the K-first layout of
``moe_gptq_gemm_rdna3`` (weights ``[E, K // 8, N]``, scales
``[E, groups, N]``), interleaves the packed nibbles per expert (the exllama
shuffle the dense RDNA3 kernel also uses) and synthesizes the symmetric
zero points the kernel dequantizes with.
"""
device = w13.device
num_experts = w13.size(0)
w13 = w13.transpose(1, 2).contiguous()
w2 = w2.transpose(1, 2).contiguous()
for e in range(num_experts):
w13_e = w13[e].contiguous()
ops.gptq_shuffle(w13_e, 4)
@@ -950,8 +953,8 @@ def _process_weights_rdna3(
return (
w13,
w2,
w13_scale.contiguous(),
w2_scale.contiguous(),
w13_scale.transpose(1, 2).contiguous(),
w2_scale.transpose(1, 2).contiguous(),
_qzeros(w13),
_qzeros(w2),
None, # w13_input_global_scale