[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499)

Signed-off-by: zixi-qi <zixi@inferact.ai>
This commit is contained in:
qizixi
2026-09-28 18:50:46 +02:00
committed by GitHub
parent 94d1462924
commit 6d3ea3c2c9
6 changed files with 357 additions and 28 deletions
@@ -120,7 +120,8 @@ Current push PP + HMA limitations:
- Only the prefiller (producer) may be PP-sharded; decode-side PP is not supported.
- Hybrid SSM/Mamba layouts are not supported under PP.
- HMA requires the same block size on P and D.
- Attention-HMA member routing requires decode TP to be no greater than prefill TP.
- Non-MLA attention-HMA layer routing requires decode TP to be no greater than prefill TP.
- Packed layouts under PP require MLA caches. Local and remote packed block strides may differ; layer pages must have equal sizes. MLA supports decode TP greater than prefill TP because its KV is replicated.
### Quantized KV cache