[Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533)

Signed-off-by: Soila Kavulya <soila.p.kavulya@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Soila Kavulya
2026-07-31 17:42:32 +00:00
committed by GitHub
co-authored by Claude
parent e67a2e0a56
commit aef85aed5d
4 changed files with 32 additions and 2 deletions
@@ -680,6 +680,7 @@ class Attention(nn.Module, AttentionLayerBase):
head_size=self.head_size,
head_size_v=self.head_size,
dtype=self.kv_cache_torch_dtype,
kv_quant_mode=quant_mode,
tq_slot_size=tq_config.slot_size_aligned,
)
else: