Encoder cache extension hooks (#48218)

Signed-off-by: hotTea <958436561@qq.com>
Signed-off-by: hanxi-java <634498162@qq.com>
Co-authored-by: hanxi-java <634498162@qq.com>
Co-authored-by: 韩熙 <63780107+hanxi-java@users.noreply.github.com>
This commit is contained in:
hotTea
2026-07-24 02:52:00 -07:00
committed by GitHub
co-authored by hanxi-java 韩熙
parent 163ecba377
commit 833483f357
8 changed files with 106 additions and 15 deletions
@@ -46,6 +46,7 @@ def _gather(features, cached, *, num_scheduled, shift, num_computed=0):
input_batch=SimpleNamespace(req_ids=["req0"]),
requests={"req0": req_state},
encoder_cache=encoder_cache,
_get_encoder_output_from_cache=lambda mm_hash: encoder_cache.get(mm_hash),
is_multimodal_pruning_enabled=False,
uses_mrope=False,
)