[Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 (#51359)
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com> Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Copilot
Wentao Ye
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
d20a031d33
commit
19d61b1273
@@ -94,6 +94,8 @@ class QuantFP8(CustomOp):
|
||||
self.is_group_quant
|
||||
and self.use_ue8m0
|
||||
and self.use_deep_gemm_supported
|
||||
and not self.static
|
||||
and self.group_size == 128
|
||||
and (DeepGemmQuantScaleFMT.from_oracle() == DeepGemmQuantScaleFMT.UE8M0)
|
||||
):
|
||||
return fp8_utils.per_token_group_quant_fp8_packed_for_deepgemm(
|
||||
|
||||
Reference in New Issue
Block a user