[Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223)
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com>
This commit is contained in:
@@ -15,8 +15,13 @@ class Mxfp8LinearLayerConfig:
|
||||
|
||||
All MXFP8 layers share the same structure: FP8-E4M3 weights with
|
||||
uint8 (E8M0) per-block scales at block size 32.
|
||||
|
||||
Attributes:
|
||||
weight_shape: The layer's `(out_features, in_features)`, i.e. `(N, K)`.
|
||||
|
||||
"""
|
||||
|
||||
weight_shape: tuple[int, int]
|
||||
bmm_batch_size: int | None = None
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user