[ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method (#49381)

Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
This commit is contained in:
Juhi Mittal
2026-09-01 16:28:56 -04:00
committed by GitHub
co-authored by Claude Opus 4.8 mgoin
parent 4bf06be985
commit 7a977c0699
6 changed files with 1173 additions and 956 deletions
@@ -38,6 +38,11 @@ from vllm.utils.platform_utils import get_device_name_as_file_name
logger = init_logger(__name__)
# Pre-fill value for scale parameters whose shards load independently. The
# shards are combined with .max(), so an unloaded shard must never win; the
# smallest representable float32 guarantees that.
FP8_SCALE_SENTINEL = torch.finfo(torch.float32).min
def is_fp8(x: torch.dtype | torch.Tensor) -> bool:
if isinstance(x, torch.Tensor):