[Docs] update attribution to reflect EDEN foundation (#41666)
Signed-off-by: amitport <1131991+amitport@users.noreply.github.com>
This commit is contained in:
@@ -5,15 +5,20 @@
|
|||||||
Hadamard rotation + per-coordinate Lloyd-Max scalar quantization for
|
Hadamard rotation + per-coordinate Lloyd-Max scalar quantization for
|
||||||
keys, uniform quantization for values.
|
keys, uniform quantization for values.
|
||||||
|
|
||||||
The technique implemented here consists of the scalar case of the HIGGS
|
The core algorithmic pattern implemented for key quantization (Hadamard
|
||||||
|
rotation followed by deterministic scalar quantization and
|
||||||
|
re-normalization) was originally established in DRIVE (Vargaftik et al.,
|
||||||
|
NeurIPS 2021) and EDEN (Vargaftik et al., ICML 2022). This formulation is
|
||||||
|
also mathematically equivalent to the scalar case of the HIGGS
|
||||||
quantization method (Malinovskii et al., "Pushing the Limits of Large
|
quantization method (Malinovskii et al., "Pushing the Limits of Large
|
||||||
Language Model Quantization via the Linearity Theorem", NAACL 2025;
|
Language Model Quantization via the Linearity Theorem", NAACL 2025;
|
||||||
preprint arXiv:2411.17525): rotation + optimized grid + optional
|
preprint arXiv:2411.17525), which subsequently generalized these concepts.
|
||||||
re-normalization, applied to KV cache compression. A first application
|
|
||||||
of this approach to KV-cache compression is in "Cache Me If You Must:
|
A first application of this approach to KV-cache compression is in "Cache
|
||||||
Adaptive Key-Value Quantization for Large Language Models" (Shutova
|
Me If You Must: Adaptive Key-Value Quantization for Large Language Models"
|
||||||
et al., ICML 2025; preprint arXiv:2501.19392). Both these references
|
(Shutova et al., ICML 2025; preprint arXiv:2501.19392). All of these
|
||||||
pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).
|
foundational and application references pre-date the TurboQuant paper
|
||||||
|
(Zandieh et al., ICLR 2026).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from vllm.model_executor.layers.quantization.turboquant.config import TurboQuantConfig
|
from vllm.model_executor.layers.quantization.turboquant.config import TurboQuantConfig
|
||||||
|
|||||||
@@ -48,19 +48,25 @@ class TurboQuantConfig:
|
|||||||
Applies Hadamard rotation followed by per-coordinate Lloyd-Max scalar
|
Applies Hadamard rotation followed by per-coordinate Lloyd-Max scalar
|
||||||
quantization for keys, and uniform quantization for values.
|
quantization for keys, and uniform quantization for values.
|
||||||
|
|
||||||
Historical note: this is the scalar case of the HIGGS quantization
|
Historical note: the core algorithmic pattern implemented for key
|
||||||
method (Malinovskii et al., "Pushing the Limits of Large Language Model
|
quantization (Hadamard rotation followed by deterministic scalar
|
||||||
Quantization via the Linearity Theorem", NAACL 2025; preprint
|
quantization and re-normalization) was originally established in DRIVE
|
||||||
arXiv:2411.17525): rotation + optimized grid + optional re-normalization,
|
(Vargaftik et al., NeurIPS 2021) and EDEN (Vargaftik et al., ICML
|
||||||
applied to KV cache compression. A first application of this approach to
|
2022). This formulation is also mathematically equivalent to the
|
||||||
KV-cache compression is in "Cache Me If You Must: Adaptive Key-Value
|
scalar case of the HIGGS quantization method (Malinovskii et al.,
|
||||||
Quantization for Large Language Models" (Shutova et al., ICML 2025;
|
"Pushing the Limits of Large Language Model Quantization via the
|
||||||
preprint arXiv:2501.19392). Both these references pre-date the
|
Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525), which
|
||||||
TurboQuant paper.
|
subsequently generalized these concepts.
|
||||||
|
|
||||||
QJL is intentionally omitted — community consensus (5+ independent
|
A first application of this approach to KV-cache compression is in
|
||||||
groups) found it hurts attention quality by amplifying variance through
|
"Cache Me If You Must: Adaptive Key-Value Quantization for Large
|
||||||
softmax.
|
Language Models" (Shutova et al., ICML 2025; preprint
|
||||||
|
arXiv:2501.19392). All of these foundational and application
|
||||||
|
references pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).
|
||||||
|
|
||||||
|
QJL is intentionally omitted: community consensus (5+ independent
|
||||||
|
groups) found it hurts attention quality by amplifying variance
|
||||||
|
through softmax.
|
||||||
|
|
||||||
Named presets (use via --kv-cache-dtype):
|
Named presets (use via --kv-cache-dtype):
|
||||||
turboquant_k8v4: FP8 keys + 4-bit values, 2.6x, +1.17% PPL
|
turboquant_k8v4: FP8 keys + 4-bit values, 2.6x, +1.17% PPL
|
||||||
|
|||||||
Reference in New Issue
Block a user