[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions (#51093)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
This commit is contained in:
@@ -519,6 +519,8 @@ class ModelOptFp8LinearMethod(LinearMethodBase):
|
||||
layer.weight, layer.weight_scale, layer.logical_widths
|
||||
)
|
||||
layer.weight = Parameter(weight.t(), requires_grad=False)
|
||||
layer.weight.input_dim = 0
|
||||
layer.weight.output_dim = 1
|
||||
layer.weight_scale = Parameter(max_w_scale, requires_grad=False)
|
||||
layer.input_scale = Parameter(layer.input_scale.max(), requires_grad=False)
|
||||
self.fp8_linear.process_weights_after_loading(layer)
|
||||
|
||||
Reference in New Issue
Block a user