[Mamba] Share RecoverSSM commit kernels and helpers across models
RecoverSSM's commit-plan and conv-window compaction kernels are model independent. Move them from models/kimi_k3 into model_executor/layers/mamba/ops/recoverssm.py (Kimi-K3 keeps its private names as import aliases; kernel source unchanged), add mamba/recoverssm_utils.py (activation predicate, argument validation) and the generic spec-row/request mapping helpers to v1/attention/backends/recoverssm_metadata.py. CacheConfig.use_kda_recoverssm becomes use_recoverssm: the switch now selects RecoverSSM for any model whose recurrent layers implement it, not only KDA. No functional change for Kimi-K3. Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,25 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
"""Model-independent RecoverSSM Python helpers: the activation predicate shared by
|
||||
layers, models and backends, and argument validation that raises ``ValueError``
|
||||
before any raw-pointer kernel launch."""
|
||||
|
||||
|
||||
def recoverssm_require(cond: bool, msg: str, component: str = "RecoverSSM") -> None:
|
||||
if not cond:
|
||||
raise ValueError(f"{component}: {msg}")
|
||||
|
||||
|
||||
def uses_recoverssm(cache_config, num_speculative_tokens: int) -> bool:
|
||||
"""True when this model instance runs RecoverSSM for its recurrent states.
|
||||
|
||||
VllmConfig sets ``cache_config.use_recoverssm`` for every model whose recurrent
|
||||
states support it (Kimi-K3 KDA, Qwen GDN/PLE). Without speculative tokens there is
|
||||
nothing to recover, so the stock path runs."""
|
||||
return (
|
||||
bool(getattr(cache_config, "use_recoverssm", False))
|
||||
and num_speculative_tokens > 0
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["recoverssm_require", "uses_recoverssm"]
|
||||
Reference in New Issue
Block a user