[KV] Restore PP fix #54793 -- no KV tensors for another stage's layers

A KV cache group with no layer on a pipeline stage was still given tensors
there, and allocate_kv_cache raised StopIteration on the second stage. The
fix was lost in the v0.30.0 rebase. Test drives the real get_kv_cache_configs
with the production layout.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 19:55:23 +07:00
co-authored by Hinata
parent 6705330924
commit e2eaefdec9
+8 -1
View File
@@ -1806,7 +1806,14 @@ def get_kv_cache_config_from_groups(
layers_by_spec: defaultdict[KVCacheSpec, list[str]] = defaultdict(list)
if isinstance(group_spec, UniformTypeKVCacheSpecs):
for layer_name, spec in group_spec.kv_cache_specs.items():
layers_by_spec[spec].append(layer_name)
# Fork (E2 PP fix #54793, dropped in the v0.30.0 rebase, restored
# the vLLM 0.30 rebase): _project_kv_cache_groups_to_worker keeps the GLOBAL
# UniformTypeKVCacheSpecs for a group with no layer on this PP
# stage, so without this filter the worker gets a tensor for
# another stage's layer and allocate_kv_cache's group lookup
# raises StopIteration (Flash-Next PP1: the layer-2 PLE state).
if layer_name in group.layer_names:
layers_by_spec[spec].append(layer_name)
elif group.layer_names:
layers_by_spec[group_spec].extend(group.layer_names)