Promote prefix_cache_retention_interval to an argument and change the default to 0 (#52216)

Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Tyler Michael Smith
2026-08-17 13:20:50 +00:00
committed by GitHub
co-authored by OpenAI Codex
parent 1d3a8b9e22
commit 017e9f4448
17 changed files with 160 additions and 63 deletions
@@ -1433,7 +1433,7 @@ class MooncakeStoreWorker:
scheduler_block_size=self.block_size,
hash_block_size=self.hash_block_size,
use_eagle=use_eagle,
retention_interval=envs.VLLM_PREFIX_CACHE_RETENTION_INTERVAL,
retention_interval=kv_cache_config.prefix_cache_retention_interval,
)
# One ChunkedTokenDatabase per group; addresses populated in
# register_kv_caches once the kv-cache layout is known. Each group's