[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
This commit is contained in:
wangxiyuan
2026-10-16 23:16:47 +00:00
committed by GitHub
co-authored by Claude Opus 5.5 Yifan Qiao
parent 2268d83bd9
commit b544587408
9 changed files with 675 additions and 49 deletions
+4
View File
@@ -2034,6 +2034,10 @@ class ModelConfig:
def has_noops(self) -> bool:
return self._model_info.has_noops
@property
def supports_kv_cache_grouping(self) -> bool:
return self._model_info.supports_kv_cache_grouping
@property
def has_inner_state(self):
return self._model_info.has_inner_state