[Doc][RL] Document the weight-transfer HTTP protocol for trainers without vLLM (#62267)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
aoshen02
2026-10-18 09:26:56 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 1b2a666391
commit 2365177ddd
6 changed files with 435 additions and 1 deletions
+8 -1
View File
@@ -145,7 +145,7 @@ engine.send_weights() # called on every rank
## API Endpoints
When running vLLM as an HTTP server, the following endpoints are available for weight transfer. `HTTPVLLMWeightSyncClient` speaks the first four for you.
When running vLLM as an HTTP server, the following endpoints are available for weight transfer. `HTTPVLLMWeightSyncClient` speaks the first five for you. A trainer may also implement the protocol directly without importing vLLM: see the wire contracts in [NCCL](nccl.md#trainer-without-vllm-over-http) and [IPC](ipc.md#trainer-without-vllm-over-http). This keeps the trainer decoupled from vLLM: it needs no vLLM install or version pin, only this documented HTTP and wire contract, so trainer and server can upgrade independently (non-PyTorch trainers such as JAX can use it too).
| Endpoint | Method | Description |
| -------- | ------ | ----------- |
@@ -153,6 +153,7 @@ When running vLLM as an HTTP server, the following endpoints are available for w
| `/start_weight_update` | POST | Start a weight update |
| `/update_weights` | POST | Transfer a batch of weights with backend-specific metadata |
| `/finish_weight_update` | POST | Finish the update and optionally commit its `weight_version` |
| `/shutdown_weight_transfer_engine` | POST | Release the transfer channel without stopping inference |
| `/update_weight_version` | POST | Update `weight_version` without changing model weights |
| `/weight_info` | GET | Get the latest committed weight version |
| `/pause` | POST | Pause generation before weight sync to handle inflight requests |
@@ -162,6 +163,12 @@ When running vLLM as an HTTP server, the following endpoints are available for w
!!! note
The HTTP weight transfer endpoints require `VLLM_SERVER_DEV_MODE=1` to be set.
!!! warning
If `/update_weights` fails, the model is left mid-reload while `/health`
stays 200, and `/finish_weight_update` is rejected. Before generating again,
resend a full `/start_weight_update` → `/update_weights` →
`/finish_weight_update` (see [#57854](https://github.com/vllm-project/vllm/pull/57854)).
The Rust frontend's optional gRPC `Control` service exposes the same pause, sleep, weight-transfer, and weight-version lifecycle for trusted sidecars. The `ServerInfo.rl_capabilities` response reports whether weight transfer and sleep mode were configured. Backend-specific `init_info` and `update_info` remain JSON metadata; model tensors continue to move over the configured NCCL, IPC, sparse-NCCL, or sharded-RDT transport.
## Extending the System