[Spec Decode][Qwen4Exp] RecoverSSM for Qwen GDN and the PLE short conv

With --use-replayssm and speculative decoding, Qwen4Exp's Gated DeltaNet layers verify the
draft window from a read-only checkpoint and keep a per-token replay record
[correction(V), normalized key(K), gate(1)] in fp32. After sampling, one commit replays the
accepted tokens and writes the recurrent state once (plus the block-boundary state in align
mode), instead of snapshotting the state after every draft position. The PLE dilated short
conv follows the same protocol (accepted-suffix compaction). num_speculative_blocks = 0,
so no per-draft Mamba blocks are reserved.

- GDN verify/commit kernels: model_executor/layers/mamba/gdn/recoverssm_gdn.py
- metadata/backends: v1/attention/backends/gdn_recoverssm.py, ple_recoverssm.py
- the verify reads strided q/k/v views of the conv output and writes in place
- builders declare AttentionCGSupport.NEVER: FULL_AND_PIECEWISE falls back to PIECEWISE;
  the fallback now also counts breakable CUDA graphs (empty splitting_ops, with or
  without torch.compile) as piecewise
  attention instead of dropping to NONE
- the generic Mamba metadata builder skips the Mamba2 ReplaySSM ring-buffer workspaces
  when --use-replayssm selects RecoverSSM (the PLE short conv uses that builder)
- tests: GDN verify/commit vs the native kernels (fp32 and bf16 state, align boundaries,
  mixed and ragged batches, extreme gates), PLE vs token-by-token decode, config selection

Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Jürgen Schmied
2026-09-27 01:17:24 +02:00
co-authored by Claude
parent a80e41d690
commit 03cae7837f
14 changed files with 2349 additions and 33 deletions
+9 -4
View File
@@ -1424,10 +1424,15 @@ class CompilationConfig:
f"with {min_cg_attn_backend} backend (support: "
f"{min_cg_support})"
)
if self.mode == CompilationMode.VLLM_COMPILE and (
self.splitting_ops_contain_attention()
or self.use_inductor_graph_partition
):
if (
self.mode == CompilationMode.VLLM_COMPILE
and (
self.splitting_ops_contain_attention()
or self.use_inductor_graph_partition
)
) or envs.VLLM_USE_BREAKABLE_CUDAGRAPH:
# Breakable CUDA graphs break at the attention kernels at runtime,
# with or without torch.compile.
msg += (
"; setting cudagraph_mode=PIECEWISE because "
"attention is compiled piecewise"