[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533)

Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Wei Zhao
2026-10-08 18:33:48 +00:00
committed by GitHub
co-authored by Wei Zhao Codex zjy0516 Claude Opus 5.5 mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 1664c4f444
commit d8fa463e2b
17 changed files with 546 additions and 200 deletions
+7 -2
View File
@@ -821,10 +821,15 @@ class Worker(WorkerBase):
# so that it's available to the warmup stage.
self.cache_config.num_gpu_blocks = kv_cache_config.num_blocks
# Adopt the engine core's layout; workers spawned after resolution
# (e.g. elastic EP scale-up) only see it through the config.
# Adopt the engine core's layout and prefix-cache granularity; workers
# spawned after resolution (e.g. elastic EP scale-up) only see them
# through the config.
if kv_cache_config.kv_cache_layout is not None:
record_kv_cache_layout(self.cache_config, kv_cache_config.kv_cache_layout)
self.cache_config.hash_block_size = kv_cache_config.hash_block_size
self.cache_config.cache_hit_alignment_tokens = (
kv_cache_config.cache_hit_alignment_tokens
)
# Init kv cache connector here, because it requires
# `kv_cache_config`.