recoverssm: RecoverSSM for Qwen GDN and the PLE short conv (port of upstream #58863, 03cae7837f)
Port of the upstream vLLM change #58863 (Apache-2.0). With --use-replayssm and speculative decoding, Gated DeltaNet layers verify the draft window from a read-only checkpoint and keep a per-token replay record; after sampling one commit replays the accepted tokens and writes the recurrent state once. The PLE short conv gets the same treatment. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -212,7 +212,7 @@ class CacheConfig:
|
||||
mamba_block_size is a multiple of replayssm_buffer_len, but this is not
|
||||
required."""
|
||||
use_recoverssm: bool = field(default=False, init=False)
|
||||
"""Whether Kimi-K3 KDA uses RecoverSSM speculative decode."""
|
||||
"""Whether RecoverSSM speculative decode is used (Kimi-K3 KDA; Qwen4Exp GDN and PLE)."""
|
||||
|
||||
# Will be set after profiling.
|
||||
num_gpu_blocks: int | None = field(default=None, init=False)
|
||||
|
||||
Reference in New Issue
Block a user