[Bugfix][ROCm] Fix FP8 per-tensor scale rank mismatch causing Inductor assertion failure (#44912)

Signed-off-by: nehmathe2 <nehmathe2@gmail.com>
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: nehmathe <nehmathe@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
nehmathe2
2026-06-16 20:17:42 -07:00
committed by GitHub
co-authored by TJian mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Divakar Verma Andreas Karatzas
parent efd15e192a
commit 2785a5e0e6
@@ -78,6 +78,12 @@ class PerTensorTorchFP8ScaledMMLinearKernel(TorchFP8ScaledMMLinearKernel):
bias: torch.Tensor | None,
output_shape: list,
) -> torch.Tensor:
# torch._scaled_mm under torch.compile does not support 0-D scales
if As.dim() == 0:
As = As.view(1)
if Bs.dim() == 0:
Bs = Bs.view(1)
output = torch._scaled_mm(
A, B, out_dtype=out_dtype, scale_a=As, scale_b=Bs, bias=bias
)