[CPU][Spec Decode] Enable DFlash SD for CPU (#44029)

Signed-off-by: guybd <guy.boudoukh@intel.com>
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
guybd
2026-06-25 15:32:48 +08:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 9b215ae60b
commit 2396d91e93
8 changed files with 266 additions and 8 deletions
+1 -1
View File
@@ -131,7 +131,7 @@ class DFlashQwen3Attention(nn.Module):
with the context K/V from the target model's hidden states. This forward op
computes attention for the query tokens only.
See also: precompute_and_store_context_kv"""
qkv = F.linear(hidden_states, self.qkv_proj.weight, self.qkv_proj.bias)
qkv, _ = self.qkv_proj(hidden_states)
q, k, v = qkv.split([self.q_size, self.kv_size, self.kv_size], dim=-1)
# Per-head RMSNorm