[CompressedTensors] FP4 Qutlass Integration (#43229)

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <brian-dellabetta@users.noreply.github.com>
Co-authored-by: Brian Dellabetta <bdellabe@redhat.com>
Co-authored-by: Brian Dellabetta <brian-dellabetta@users.noreply.github.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
This commit is contained in:
Kyle Sayers
2026-07-28 20:34:20 -06:00
committed by GitHub
co-authored by Brian Dellabetta Brian Dellabetta Dipika Sikka
parent 7398a30d79
commit 30c2718eaa
7 changed files with 169 additions and 34 deletions
@@ -473,6 +473,11 @@ def test_compressed_tensors_w4a8_fp8(vllm_runner, args):
"Flat is better than nested.\nSparse is better than dense.",
150.0,
),
(
"nm-testing/Llama-3.2-1B-Instruct-quipv16-nvfp4",
"Flat is better than nested.\nSparse is better than dense.",
150.0,
),
],
)
def test_compressed_tensors_transforms_perplexity(