[Attention] Re-enable cross-layer KV cache layout for MLA via stride-aware kernels (#45111)

Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
This commit is contained in:
Yifan Qiao
2026-06-22 06:57:02 -07:00
committed by GitHub
parent a4610da0c6
commit aa4990a9a2
13 changed files with 847 additions and 21 deletions
@@ -1212,9 +1212,9 @@ class MLACommonBackend(AttentionBackend):
include_num_layers_dimension: bool = False,
) -> tuple[int, ...]:
if include_num_layers_dimension:
# MLA kernels require contiguous per-layer KV cache views.
# Identity permutation keeps num_layers first in physical
# layout, signaling cross-layer allocation is unsupported.
# Default to identity permutation to signal cross-layer allocation
# is unsupported. Each MLA backend must opt in to support cross-layer
# allocation by overriding this method.
return (0, 1, 2, 3)
return (0, 1, 2)