[RL] Add rank-local IPC weight updates (#52497)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
aoshen02
2026-08-26 15:36:57 +08:00
committed by GitHub
co-authored by OpenAI Codex OpenAI Codex Roger Wang
parent a5f82882a6
commit ee5ae2ae70
6 changed files with 80 additions and 8 deletions
+6
View File
@@ -110,6 +110,12 @@ mode carries a per-chunk barrier across ranks: without it, a rank could overwrit
its buffer while its colocated worker is still reading the current chunk. This is
handled inside `send_weights()`.
## Rank-Local Updates
For different parameter subsets per worker, `update_info` may be a list indexed
by worker rank. This form is IPC-only because collective backends require every
worker to participate.
## Examples
- [RLHF with IPC weight syncing (`vllm serve`, HTTP)](../../../examples/rl/rlhf_http_ipc.py) - **Start here.** Server and training model share a single GPU; HTTP control plane, CUDA IPC data plane. Launches and tears down its own server