[Core][Frontend] Add weight version tagging for RL rollouts (#49040)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> Signed-off-by: Shuolei Wang <948904026@qq.com>
This commit is contained in:
@@ -65,6 +65,8 @@ For further details on Weight Transfer, please refer to [this page](../training/
|
||||
- `LLM.start_weight_update` - Starts a new weight update cycle.
|
||||
- `LLM.update_weights` - Updates the model weights.
|
||||
- `LLM.finish_weight_update` - Finishes the current weight update cycle.
|
||||
- `LLM.update_weight_version` - Sets the weight version without updating model weights.
|
||||
- `LLM.get_weight_version` - Returns the latest committed weight version.
|
||||
|
||||
## Additional APIs
|
||||
|
||||
|
||||
@@ -179,6 +179,8 @@ For further details on Weight Transfer, please refer to [this page](../../traini
|
||||
- `/start_weight_update` - Prepares the inference engine for a weight update.
|
||||
- `/update_weights` - Update model weights (can alter model behavior)
|
||||
- `/finish_weight_update` - Finalizes the weight update
|
||||
- `/update_weight_version` - Set the weight version without updating model weights
|
||||
- `/weight_info` - Get the latest committed weight version
|
||||
- `/get_world_size` - Get distributed world size
|
||||
|
||||
### Collective RPC
|
||||
|
||||
@@ -38,11 +38,12 @@ Resumes the scheduler after a pause. Any requests frozen with `mode="keep"` will
|
||||
|
||||
### HTTP Endpoints
|
||||
|
||||
When using the vLLM HTTP server, the same functionality is available via:
|
||||
With `VLLM_SERVER_DEV_MODE=1`, the vLLM HTTP server exposes the same functionality via:
|
||||
|
||||
- `POST /pause?mode=keep` - Pause generation
|
||||
- `POST /resume` - Resume generation
|
||||
- `POST /abort_requests` - Abort in-flight requests without pausing the scheduler (send `{}` to abort all, or `{"request_ids": [...]}`)
|
||||
- `GET /weight_info` - Return the latest committed `weight_version`
|
||||
|
||||
!!! note "Data Parallelism"
|
||||
When using data parallelism with vLLM's **internal load balancer** (i.e. `data_parallel_backend="ray"`), pause and resume are handled automatically across all DP ranks -- a single call is sufficient. When using an **external load balancer** (i.e. multiple independent vLLM instances behind a proxy), you must send pause and resume requests to **every** engine instance individually before and after the weight update.
|
||||
|
||||
@@ -53,7 +53,9 @@ When running vLLM as an HTTP server, the following endpoints are available for w
|
||||
| `/init_weight_transfer_engine` | POST | Initialize the weight transfer engine with backend-specific info |
|
||||
| `/start_weight_update` | POST | Start a weight update |
|
||||
| `/update_weights` | POST | Transfer a batch of weights with backend-specific metadata |
|
||||
| `/finish_weight_update` | POST | Finish the weight update and run post-processing |
|
||||
| `/finish_weight_update` | POST | Finish the update and optionally commit its `weight_version` |
|
||||
| `/update_weight_version` | POST | Update `weight_version` without changing model weights |
|
||||
| `/weight_info` | GET | Get the latest committed weight version |
|
||||
| `/pause` | POST | Pause generation before weight sync to handle inflight requests |
|
||||
| `/resume` | POST | Resume generation after weight sync |
|
||||
| `/get_world_size` | GET | Get the number of inference workers (useful for NCCL world size calculation) |
|
||||
@@ -79,7 +81,7 @@ EngineClass.trainer_send_weights(
|
||||
)
|
||||
|
||||
# 4. Finish weight update on inference side
|
||||
llm.finish_weight_update()
|
||||
llm.finish_weight_update(weight_version="step-42")
|
||||
```
|
||||
|
||||
See the [NCCL](nccl.md) and [IPC](ipc.md) pages for backend-specific trainer APIs and full examples.
|
||||
|
||||
Reference in New Issue
Block a user