[Feature][Humming] Humming feature integration (#56685)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: mgoin <mgoin64@gmail.com>
This commit is contained in:
co-authored by
Codex
mgoin
parent
27757dde02
commit
7d99c2c4fd
@@ -13,8 +13,8 @@ from vllm.model_executor.layers.quantization.utils.quant_utils import QuantKey
|
||||
class MxFp6LinearLayerConfig:
|
||||
"""Configuration for an MXFP6 linear layer.
|
||||
|
||||
All MXFP6 layers share the same structure: packed uint8 weights (2 FP4 values per
|
||||
byte) and per-block weight scales (group size 32).
|
||||
All MXFP6 layers share the same structure: packed uint8 weights (4 FP6 values
|
||||
per 3 bytes) and per-block weight scales (group size 32).
|
||||
|
||||
Attributes:
|
||||
weight_quant_key: Identifies the weight quantization format. Can be
|
||||
|
||||
Reference in New Issue
Block a user