[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
This commit is contained in:
co-authored by
Claude Opus 5.5
Yifan Qiao
parent
2268d83bd9
commit
b544587408
@@ -2034,6 +2034,10 @@ class ModelConfig:
|
||||
def has_noops(self) -> bool:
|
||||
return self._model_info.has_noops
|
||||
|
||||
@property
|
||||
def supports_kv_cache_grouping(self) -> bool:
|
||||
return self._model_info.supports_kv_cache_grouping
|
||||
|
||||
@property
|
||||
def has_inner_state(self):
|
||||
return self._model_info.has_inner_state
|
||||
|
||||
Reference in New Issue
Block a user