[KV] Restore PP fix #54795 -- PP stages need only share a KV layout

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 19:55:23 +07:00
co-authored by Hinata
parent e2eaefdec9
commit 09ea3a66e9
+17 -6
View File
@@ -264,8 +264,9 @@ def resolve_kv_cache_layout(
"""Resolve one KV cache layout for the whole model.
Runs once in the engine core. Every worker reports the layouts its backends
support, most preferred first (``get_supported_kv_cache_layouts``); all
ranks run the same backends, so their lists must agree. Specs mixing HNC
support, most preferred first (``get_supported_kv_cache_layouts``); ranks
may expose different sets under pipeline parallelism, but their lists must
share at least one layout. Specs mixing HNC
shapes narrow the candidates to block-compact layouts. An explicit
``VLLM_KV_CACHE_LAYOUT`` must be one of the candidates or resolution fails,
with the legacy ``NHD``/``HND`` names as aliases for ``LBNHC``/``LBHNC``; the
@@ -282,10 +283,20 @@ def resolve_kv_cache_layout(
assert supported_layouts and all(supported_layouts), (
"No worker reported supported KV cache layouts."
)
assert all(names == supported_layouts[0] for names in supported_layouts[1:]), (
f"Workers disagree on supported KV cache layouts: {supported_layouts}."
)
candidates = [_layout_from_name(name) for name in supported_layouts[0]]
# Fork: restore PP fix #54795 from the E2 base, dropped
# in the v0.30.0 rebase. Under pipeline parallelism the stages run
# different backends (PP0 carries the PLE short-conv, PP1 the drafter), so
# their lists need only share a layout, not be identical.
common_layout_names = [
name
for name in supported_layouts[0]
if all(name in names for names in supported_layouts[1:])
]
if not common_layout_names:
raise ValueError(
f"Workers share no supported KV cache layout: {supported_layouts}."
)
candidates = [_layout_from_name(name) for name in common_layout_names]
# A block-compact layout means the block is densely packed in memory, so any mix of
# specs can re-interpret HNC with different sizes as long as the total number of