recoverssm: --use-replayssm without MTP falls back to the stock GDN path

On Qwen4Exp the flag selects RecoverSSM, which needs speculative decoding. With num_speculative_tokens=0 the
validator refused to start; it now logs a warning, clears use_replayssm (the no-flag path) and continues, so one
launch line serves with and without MTP. Other architectures keep the error. Config tests cover both.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 20:09:17 +07:00
co-authored by Hinata
parent 8c518d4f09
commit 7c0a916c5d
+10 -4
View File
@@ -3530,11 +3530,17 @@ class VllmConfig:
and not self.cache_config.use_recoverssm
):
# Qwen4Exp's GDN and PLE layers have no plain ReplaySSM path; with them the
# flag selects RecoverSSM, which needs speculative decoding.
raise ValueError(
"--use-replayssm on Qwen4Exp selects GDN RecoverSSM and requires "
"speculative decoding (num_speculative_tokens > 0)"
# flag selects RecoverSSM, which needs speculative decoding. Fork (v2.5):
# without speculative decoding there are no drafts to recover from, so the
# flag falls back to the stock path (as if it were not passed) instead of
# refusing to start; one launch line then serves with and without MTP.
logger.warning(
"--use-replayssm on Qwen4Exp selects GDN RecoverSSM, which needs "
"speculative decoding; num_speculative_tokens=0, so the stock GDN "
"path is used (as without --use-replayssm)"
)
self.cache_config.use_replayssm = False
return self
if (
self.mamba_config.backend == MambaBackendEnum.FLASHINFER
and self.cache_config.replayssm_buffer_len > 16