[Bugfix][ROCm] Fix FP8 per-tensor scale rank mismatch causing Inductor assertion failure (#44912)
Signed-off-by: nehmathe2 <nehmathe2@gmail.com> Signed-off-by: Divakar Verma <divakar.verma@amd.com> Signed-off-by: nehmathe <nehmathe@amd.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Divakar Verma <divakar.verma@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
co-authored by
TJian
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Divakar Verma
Andreas Karatzas
parent
efd15e192a
commit
2785a5e0e6
@@ -78,6 +78,12 @@ class PerTensorTorchFP8ScaledMMLinearKernel(TorchFP8ScaledMMLinearKernel):
|
||||
bias: torch.Tensor | None,
|
||||
output_shape: list,
|
||||
) -> torch.Tensor:
|
||||
# torch._scaled_mm under torch.compile does not support 0-D scales
|
||||
if As.dim() == 0:
|
||||
As = As.view(1)
|
||||
if Bs.dim() == 0:
|
||||
Bs = Bs.view(1)
|
||||
|
||||
output = torch._scaled_mm(
|
||||
A, B, out_dtype=out_dtype, scale_a=As, scale_b=Bs, bias=bias
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user