recoverssm: --use-replayssm without MTP falls back to the stock GDN path
On Qwen4Exp the flag selects RecoverSSM, which needs speculative decoding. With num_speculative_tokens=0 the validator refused to start; it now logs a warning, clears use_replayssm (the no-flag path) and continues, so one launch line serves with and without MTP. Other architectures keep the error. Config tests cover both. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
+10
-4
@@ -3530,11 +3530,17 @@ class VllmConfig:
|
||||
and not self.cache_config.use_recoverssm
|
||||
):
|
||||
# Qwen4Exp's GDN and PLE layers have no plain ReplaySSM path; with them the
|
||||
# flag selects RecoverSSM, which needs speculative decoding.
|
||||
raise ValueError(
|
||||
"--use-replayssm on Qwen4Exp selects GDN RecoverSSM and requires "
|
||||
"speculative decoding (num_speculative_tokens > 0)"
|
||||
# flag selects RecoverSSM, which needs speculative decoding. Fork (v2.5):
|
||||
# without speculative decoding there are no drafts to recover from, so the
|
||||
# flag falls back to the stock path (as if it were not passed) instead of
|
||||
# refusing to start; one launch line then serves with and without MTP.
|
||||
logger.warning(
|
||||
"--use-replayssm on Qwen4Exp selects GDN RecoverSSM, which needs "
|
||||
"speculative decoding; num_speculative_tokens=0, so the stock GDN "
|
||||
"path is used (as without --use-replayssm)"
|
||||
)
|
||||
self.cache_config.use_replayssm = False
|
||||
return self
|
||||
if (
|
||||
self.mamba_config.backend == MambaBackendEnum.FLASHINFER
|
||||
and self.cache_config.replayssm_buffer_len > 16
|
||||
|
||||
Reference in New Issue
Block a user