[Core][Frontend] Add weight version tagging for RL rollouts (#49040)

Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
This commit is contained in:
Shuolei Wang
2026-07-28 14:35:29 +08:00
committed by GitHub
parent 90245f4190
commit 9069a57139
16 changed files with 142 additions and 18 deletions
+2 -1
View File
@@ -38,11 +38,12 @@ Resumes the scheduler after a pause. Any requests frozen with `mode="keep"` will
### HTTP Endpoints
When using the vLLM HTTP server, the same functionality is available via:
With `VLLM_SERVER_DEV_MODE=1`, the vLLM HTTP server exposes the same functionality via:
- `POST /pause?mode=keep` - Pause generation
- `POST /resume` - Resume generation
- `POST /abort_requests` - Abort in-flight requests without pausing the scheduler (send `{}` to abort all, or `{"request_ids": [...]}`)
- `GET /weight_info` - Return the latest committed `weight_version`
!!! note "Data Parallelism"
When using data parallelism with vLLM's **internal load balancer** (i.e. `data_parallel_backend="ray"`), pause and resume are handled automatically across all DP ranks -- a single call is sufficient. When using an **external load balancer** (i.e. multiple independent vLLM instances behind a proxy), you must send pause and resume requests to **every** engine instance individually before and after the weight update.