Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Chao Lei <leichao139636@163.com> Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Chao Lei <leichao139636@163.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
Chao Lei
Claude Fable 5
Matthew Bonanni
OpenAI Codex
OpenAI Codex
parent
ebe1dec2da
commit
e19a3e172e
@@ -60,6 +60,16 @@ sees only device pools.
|
||||
The source group has `block_pool_id=None`; device-pool consumers must narrow it
|
||||
before indexing, so host ownership cannot masquerade as a numeric GPU pool.
|
||||
|
||||
For single-node MP tensor parallelism, every TP worker maps the same pinned host
|
||||
pool and uses the same block and layer offsets. MLA source KV is replicated
|
||||
across TP ranks, so this stores one physical copy instead of one copy per rank.
|
||||
TP rank 0 writes the shared host pool; peers wait on its IPC events before
|
||||
reading it. Other executor and parallel layouts retain private per-rank pools.
|
||||
|
||||
The shared layout backs the per-replica logical capacity with one physical pool;
|
||||
the private layout allocates one physical pool per rank. Physical pool size
|
||||
includes block-stride alignment.
|
||||
|
||||
## Code boundary
|
||||
|
||||
```text
|
||||
|
||||
Reference in New Issue
Block a user