[5/N] Share HiSparse host cache across TP ranks (reopens #52760) (#56629)

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chao Lei <leichao139636@163.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Chao Lei <leichao139636@163.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Lucas Wilkinson
2026-09-12 19:48:25 -04:00
committed by GitHub
co-authored by Chao Lei Claude Fable 5 Matthew Bonanni OpenAI Codex OpenAI Codex
parent ebe1dec2da
commit e19a3e172e
14 changed files with 878 additions and 85 deletions
+10
View File
@@ -60,6 +60,16 @@ sees only device pools.
The source group has `block_pool_id=None`; device-pool consumers must narrow it
before indexing, so host ownership cannot masquerade as a numeric GPU pool.
For single-node MP tensor parallelism, every TP worker maps the same pinned host
pool and uses the same block and layer offsets. MLA source KV is replicated
across TP ranks, so this stores one physical copy instead of one copy per rank.
TP rank 0 writes the shared host pool; peers wait on its IPC events before
reading it. Other executor and parallel layouts retain private per-rank pools.
The shared layout backs the per-replica logical capacity with one physical pool;
the private layout allocates one physical pool per rank. Physical pool size
includes block-stride alignment.
## Code boundary
```text