[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863)

Intake of open upstream PR vllm-project/vllm#58863. Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.

Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.

RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.

Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-09-29 13:39:09 +07:00
co-authored by Hinata
31 changed files with 2777 additions and 226 deletions
+4 -4
View File
@@ -261,7 +261,7 @@ def _create_decode_vllm_config(
num_speculative_tokens: int = 0,
dynamic_spec_schedule: list[tuple[int, int, int]] | None = None,
max_num_seqs: int = 8,
use_kda_recoverssm: bool = False,
use_recoverssm: bool = False,
) -> MagicMock:
compilation_config = CompilationConfig(
cudagraph_mode="FULL_AND_PIECEWISE",
@@ -276,7 +276,7 @@ def _create_decode_vllm_config(
max_num_seqs=max_num_seqs
)
vllm_config.parallel_config = ParallelConfig()
vllm_config.cache_config = SimpleNamespace(use_kda_recoverssm=use_kda_recoverssm)
vllm_config.cache_config = SimpleNamespace(use_recoverssm=use_recoverssm)
vllm_config.num_speculative_tokens = num_speculative_tokens
if dynamic_spec_schedule is None:
vllm_config.speculative_config = None
@@ -295,7 +295,7 @@ def _make_spec_decode_manager(
num_speculative_tokens: int = 0,
dynamic_spec_schedule: list[tuple[int, int, int]] | None = None,
max_num_seqs: int = 8,
use_kda_recoverssm: bool = False,
use_recoverssm: bool = False,
varlen_decode: bool = False,
) -> gpu_cudagraph_utils.CudaGraphManager:
monkeypatch.setattr(
@@ -314,7 +314,7 @@ def _make_spec_decode_manager(
num_speculative_tokens=num_speculative_tokens,
dynamic_spec_schedule=dynamic_spec_schedule,
max_num_seqs=max_num_seqs,
use_kda_recoverssm=use_kda_recoverssm,
use_recoverssm=use_recoverssm,
),
device=torch.device("cpu"),
cudagraph_mode=CUDAGraphMode.FULL_AND_PIECEWISE,