[Refactor] Remove Fp8OnlineLinearMethod as scheduled (#45463)
Signed-off-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
d53f4593ce
commit
3f53e2138f
@@ -28,9 +28,9 @@ For more information on implementation, see [Low Level `layerwise` API](#low-lev
|
||||
Online quantization refers to when a user provides full precision weights and those weights are quantized on-the-fly as they are loaded into the model. The layerwise reloading system handles this by treating online quantization as a **processing** step, which is then handled in an online way both during first-time load and during reload. A typical online quantization method implementation should look like this:
|
||||
|
||||
```python
|
||||
class Fp8OnlineLinearMethod(Fp8LinearMethod):
|
||||
"""Online version of Fp8LinearMethod which loads a full precision checkpoint
|
||||
and quantizes weights during loading."""
|
||||
class Fp8PerTensorOnlineLinearMethod(LinearMethodBase):
|
||||
"""Online version of FP8 per-tensor quantization which loads a full
|
||||
precision checkpoint and quantizes weights during loading."""
|
||||
|
||||
uses_meta_device: bool = True
|
||||
|
||||
|
||||
Reference in New Issue
Block a user