[Bugfix][Spec Decode] Fix DFlash draft/target layer-count mismatch (#48113)

Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Nick Hill
2026-07-10 04:42:53 -07:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 68ea76e780
commit 7614b88ebd
2 changed files with 9 additions and 1 deletions
@@ -754,6 +754,14 @@ class DFlashQwen3ForCausalLM(Qwen3ForCausalLM):
needs_squeeze = hidden_states.dim() == 1
if needs_squeeze:
hidden_states = hidden_states.unsqueeze(0)
expected = self.model.fc.input_size
if hidden_states.shape[-1] != expected:
raise ValueError(
f"DFlash drafter expects {expected} concatenated aux hidden "
f"features but received {hidden_states.shape[-1]}. This usually "
"means the draft model's target_layer_ids reference layers that "
"do not exist in the target model (incompatible draft/target pair)."
)
result = self.model.fc(hidden_states)
if needs_squeeze:
result = result.squeeze(0)