[XPU] properly handle q_descale on XPU as quant query input not supported (#39676)
Signed-off-by: Yan Ma <yan.ma@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
This commit is contained in:
@@ -1031,7 +1031,9 @@ class FlashAttentionImpl(AttentionImpl):
|
||||
window_size=sliding_window_size,
|
||||
softcap=self.logits_soft_cap,
|
||||
fa_version=self.vllm_flash_attn_version,
|
||||
q_descale=layer._q_scale.expand(descale_shape),
|
||||
q_descale=layer._q_scale.expand(descale_shape)
|
||||
if self.supports_quant_query_input
|
||||
else None,
|
||||
k_descale=layer._k_scale.expand(descale_shape),
|
||||
v_descale=layer._v_scale.expand(descale_shape),
|
||||
num_splits=1 if self.batch_invariant_enabled else 0,
|
||||
|
||||
Reference in New Issue
Block a user