[Core][Frontend] Add weight version tagging for RL rollouts (#49040)

Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
This commit is contained in:
Shuolei Wang
2026-07-28 14:35:29 +08:00
committed by GitHub
parent 90245f4190
commit 9069a57139
16 changed files with 142 additions and 18 deletions
+2
View File
@@ -65,6 +65,8 @@ For further details on Weight Transfer, please refer to [this page](../training/
- `LLM.start_weight_update` - Starts a new weight update cycle.
- `LLM.update_weights` - Updates the model weights.
- `LLM.finish_weight_update` - Finishes the current weight update cycle.
- `LLM.update_weight_version` - Sets the weight version without updating model weights.
- `LLM.get_weight_version` - Returns the latest committed weight version.
## Additional APIs
+2
View File
@@ -179,6 +179,8 @@ For further details on Weight Transfer, please refer to [this page](../../traini
- `/start_weight_update` - Prepares the inference engine for a weight update.
- `/update_weights` - Update model weights (can alter model behavior)
- `/finish_weight_update` - Finalizes the weight update
- `/update_weight_version` - Set the weight version without updating model weights
- `/weight_info` - Get the latest committed weight version
- `/get_world_size` - Get distributed world size
### Collective RPC
+2 -1
View File
@@ -38,11 +38,12 @@ Resumes the scheduler after a pause. Any requests frozen with `mode="keep"` will
### HTTP Endpoints
When using the vLLM HTTP server, the same functionality is available via:
With `VLLM_SERVER_DEV_MODE=1`, the vLLM HTTP server exposes the same functionality via:
- `POST /pause?mode=keep` - Pause generation
- `POST /resume` - Resume generation
- `POST /abort_requests` - Abort in-flight requests without pausing the scheduler (send `{}` to abort all, or `{"request_ids": [...]}`)
- `GET /weight_info` - Return the latest committed `weight_version`
!!! note "Data Parallelism"
When using data parallelism with vLLM's **internal load balancer** (i.e. `data_parallel_backend="ray"`), pause and resume are handled automatically across all DP ranks -- a single call is sufficient. When using an **external load balancer** (i.e. multiple independent vLLM instances behind a proxy), you must send pause and resume requests to **every** engine instance individually before and after the weight update.
+4 -2
View File
@@ -53,7 +53,9 @@ When running vLLM as an HTTP server, the following endpoints are available for w
| `/init_weight_transfer_engine` | POST | Initialize the weight transfer engine with backend-specific info |
| `/start_weight_update` | POST | Start a weight update |
| `/update_weights` | POST | Transfer a batch of weights with backend-specific metadata |
| `/finish_weight_update` | POST | Finish the weight update and run post-processing |
| `/finish_weight_update` | POST | Finish the update and optionally commit its `weight_version` |
| `/update_weight_version` | POST | Update `weight_version` without changing model weights |
| `/weight_info` | GET | Get the latest committed weight version |
| `/pause` | POST | Pause generation before weight sync to handle inflight requests |
| `/resume` | POST | Resume generation after weight sync |
| `/get_world_size` | GET | Get the number of inference workers (useful for NCCL world size calculation) |
@@ -79,7 +81,7 @@ EngineClass.trainer_send_weights(
)
# 4. Finish weight update on inference side
llm.finish_weight_update()
llm.finish_weight_update(weight_version="step-42")
```
See the [NCCL](nccl.md) and [IPC](ipc.md) pages for backend-specific trainer APIs and full examples.