iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[BUGFIX][Quant]Fix test_kv_scale_reload failed (#50405)

Browse Source 
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Yejing Lai
2026-08-05 10:50:54 +08:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent c416f15710
commit 2cb3ff88cc
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a16_fp8.py
+3 -2
View File
@@ -137,8 +137,9 @@ class CompressedTensorsW8A16Fp8(CompressedTensorsScheme):
"weight_scale",
convert_to_channelwise(layer.weight_scale, layer.logical_widths),
)
self.strategy = QuantizationStrategy.CHANNEL
self.weight_quant_key = STRATEGY_TO_WEIGHT_QUANT_KEY[self.strategy]
self.weight_quant_key = STRATEGY_TO_WEIGHT_QUANT_KEY[
QuantizationStrategy.CHANNEL
]
self.linear_kernel.config.weight_quant_key = self.weight_quant_key
# Canonicalize to (K, N) for the kernel.