[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863)
Intake of open upstream PR vllm-project/vllm#58863. Active only with VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise. Speculative decoding on Gated DeltaNet snapshotted the full recurrent state after every draft position and reserved one Mamba block per draft position. On a 24 GB card that is reserved VRAM we never use, plus per-step copy traffic. RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this: verify runs from a read-only checkpoint and stores a small per-token replay record [correction (V), normalized key (K), gate (1)] in fp32; commit replays only the accepted tokens and writes the recurrent state once, plus the block-boundary state in align mode. The PLE dilated short conv compacts its accepted window with the same plan. Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens() work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM to the same base list. Kept both. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -151,7 +151,7 @@ def test_kda_recoverssm_derivation_is_revalidated():
|
||||
config = SimpleNamespace(
|
||||
cache_config=SimpleNamespace(
|
||||
use_replayssm=True,
|
||||
use_kda_recoverssm=False,
|
||||
use_recoverssm=False,
|
||||
mamba_cache_mode="none",
|
||||
),
|
||||
num_speculative_tokens=3,
|
||||
@@ -170,7 +170,7 @@ def test_kda_recoverssm_derivation_is_revalidated():
|
||||
|
||||
VllmConfig.validate_mamba_cached_kernel(config)
|
||||
assert config.cache_config.use_replayssm
|
||||
assert config.cache_config.use_kda_recoverssm
|
||||
assert config.cache_config.use_recoverssm
|
||||
|
||||
config.cache_config.mamba_cache_mode = "align"
|
||||
VllmConfig.validate_mamba_cached_kernel(config)
|
||||
|
||||
Reference in New Issue
Block a user