[Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225)

This commit is contained in:
Michael Goin
2024-08-06 18:34:26 -07:00
committed by GitHub
parent fd95e026e0
commit f9a5600649
4 changed files with 33 additions and 14 deletions
@@ -251,7 +251,6 @@ class GPTQMarlinLinearMethod(LinearMethodBase):
scales_and_zp_size,
output_size_per_partition // self.quant_config.pack_factor,
dtype=torch.int32,
device="meta",
),
requires_grad=False,
)