[ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008)
Signed-off-by: xuebwang-amd <xuebwang@amd.com> Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
This commit is contained in:
co-authored by
Robert Shaw
Robert Shaw
parent
599e4335a4
commit
b129136c7a
@@ -168,3 +168,19 @@ class QuantizationConfig(ABC):
|
||||
Interface to update values after config initialization.
|
||||
"""
|
||||
pass
|
||||
|
||||
def is_mxfp4_quant(self, prefix: str, layer: torch.nn.Module) -> bool:
|
||||
"""
|
||||
Determine if mxfp4 quantization will be used for this config.
|
||||
|
||||
This allows hidden_size rounding to happen before moe_config creation
|
||||
without needing to instantiate quant_method first.
|
||||
|
||||
Args:
|
||||
prefix: The layer prefix/name in the model
|
||||
layer: The layer module
|
||||
|
||||
Returns:
|
||||
True if this config uses MXFP4 quantization, False otherwise
|
||||
"""
|
||||
return False
|
||||
|
||||
Reference in New Issue
Block a user