[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
This commit is contained in:
wangxiyuan
2026-10-16 23:16:47 +00:00
committed by GitHub
co-authored by Claude Opus 5.5 Yifan Qiao
parent 2268d83bd9
commit b544587408
9 changed files with 675 additions and 49 deletions
+22
View File
@@ -1467,6 +1467,28 @@ class Platform:
"Unknown vLLM environment variable detected: %s", env
)
@classmethod
def get_kv_cache_config_builder_cls(cls, vllm_config: "VllmConfig") -> str:
"""Resolve the fully-qualified class path of the platform's KV cache
config builder.
.. note::
Experimental, like the ``KVCacheConfigBuilder`` interface it selects.
The builder supplies grouping (``get_kv_cache_groups``) and tensor
placement (``get_kv_cache_config_from_groups``). Vendors may override
this to return a platform-specific builder. The default builder honors a
model implementing
:class:`~vllm.model_executor.models.interfaces.SupportsKVCacheGrouping`;
a builder that overrides ``get_kv_cache_groups`` owns grouping and
bypasses that model hook unless it calls ``super()``.
Returns:
Fully-qualified class path of a ``KVCacheConfigBuilder`` subclass.
"""
return "vllm.v1.core.kv_cache_planning.DefaultKVCacheConfigBuilder"
class UnspecifiedPlatform(Platform):
_enum = PlatformEnum.UNSPECIFIED