[Docs] update attribution to reflect EDEN foundation (#41666)

Signed-off-by: amitport <1131991+amitport@users.noreply.github.com>
This commit is contained in:
Amit Portnoy
2026-05-18 11:22:56 -07:00
committed by GitHub
parent 00e20e76f7
commit ce88f01c9a
2 changed files with 30 additions and 19 deletions
@@ -5,15 +5,20 @@
Hadamard rotation + per-coordinate Lloyd-Max scalar quantization for Hadamard rotation + per-coordinate Lloyd-Max scalar quantization for
keys, uniform quantization for values. keys, uniform quantization for values.
The technique implemented here consists of the scalar case of the HIGGS The core algorithmic pattern implemented for key quantization (Hadamard
rotation followed by deterministic scalar quantization and
re-normalization) was originally established in DRIVE (Vargaftik et al.,
NeurIPS 2021) and EDEN (Vargaftik et al., ICML 2022). This formulation is
also mathematically equivalent to the scalar case of the HIGGS
quantization method (Malinovskii et al., "Pushing the Limits of Large quantization method (Malinovskii et al., "Pushing the Limits of Large
Language Model Quantization via the Linearity Theorem", NAACL 2025; Language Model Quantization via the Linearity Theorem", NAACL 2025;
preprint arXiv:2411.17525): rotation + optimized grid + optional preprint arXiv:2411.17525), which subsequently generalized these concepts.
re-normalization, applied to KV cache compression. A first application
of this approach to KV-cache compression is in "Cache Me If You Must: A first application of this approach to KV-cache compression is in "Cache
Adaptive Key-Value Quantization for Large Language Models" (Shutova Me If You Must: Adaptive Key-Value Quantization for Large Language Models"
et al., ICML 2025; preprint arXiv:2501.19392). Both these references (Shutova et al., ICML 2025; preprint arXiv:2501.19392). All of these
pre-date the TurboQuant paper (Zandieh et al., ICLR 2026). foundational and application references pre-date the TurboQuant paper
(Zandieh et al., ICLR 2026).
""" """
from vllm.model_executor.layers.quantization.turboquant.config import TurboQuantConfig from vllm.model_executor.layers.quantization.turboquant.config import TurboQuantConfig
@@ -48,19 +48,25 @@ class TurboQuantConfig:
Applies Hadamard rotation followed by per-coordinate Lloyd-Max scalar Applies Hadamard rotation followed by per-coordinate Lloyd-Max scalar
quantization for keys, and uniform quantization for values. quantization for keys, and uniform quantization for values.
Historical note: this is the scalar case of the HIGGS quantization Historical note: the core algorithmic pattern implemented for key
method (Malinovskii et al., "Pushing the Limits of Large Language Model quantization (Hadamard rotation followed by deterministic scalar
Quantization via the Linearity Theorem", NAACL 2025; preprint quantization and re-normalization) was originally established in DRIVE
arXiv:2411.17525): rotation + optimized grid + optional re-normalization, (Vargaftik et al., NeurIPS 2021) and EDEN (Vargaftik et al., ICML
applied to KV cache compression. A first application of this approach to 2022). This formulation is also mathematically equivalent to the
KV-cache compression is in "Cache Me If You Must: Adaptive Key-Value scalar case of the HIGGS quantization method (Malinovskii et al.,
Quantization for Large Language Models" (Shutova et al., ICML 2025; "Pushing the Limits of Large Language Model Quantization via the
preprint arXiv:2501.19392). Both these references pre-date the Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525), which
TurboQuant paper. subsequently generalized these concepts.
QJL is intentionally omitted — community consensus (5+ independent A first application of this approach to KV-cache compression is in
groups) found it hurts attention quality by amplifying variance through "Cache Me If You Must: Adaptive Key-Value Quantization for Large
softmax. Language Models" (Shutova et al., ICML 2025; preprint
arXiv:2501.19392). All of these foundational and application
references pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).
QJL is intentionally omitted: community consensus (5+ independent
groups) found it hurts attention quality by amplifying variance
through softmax.
Named presets (use via --kv-cache-dtype): Named presets (use via --kv-cache-dtype):
turboquant_k8v4: FP8 keys + 4-bit values, 2.6x, +1.17% PPL turboquant_k8v4: FP8 keys + 4-bit values, 2.6x, +1.17% PPL