[Frontend] Add /abort_requests to the RLHF dev API router (#47173)
Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
a02984ed47
commit
83762b77b0
@@ -170,6 +170,7 @@ For further details on Weight Transfer, please refer to [this page](../../traini
|
||||
- `/pause` - Pause generation (causes denial of service)
|
||||
- `/resume` - Resume generation
|
||||
- `/is_paused` - Check if generation is paused
|
||||
- `/abort_requests` - Abort in-flight requests (all in-flight, or the given `request_ids`) without pausing the scheduler
|
||||
- `/init_weight_transfer_engine` - Initialize weight transfer engine for RLHF
|
||||
- `/start_weight_update` - Prepares the inference engine for a weight update.
|
||||
- `/update_weights` - Update model weights (can alter model behavior)
|
||||
|
||||
@@ -42,6 +42,7 @@ When using the vLLM HTTP server, the same functionality is available via:
|
||||
|
||||
- `POST /pause?mode=keep` - Pause generation
|
||||
- `POST /resume` - Resume generation
|
||||
- `POST /abort_requests` - Abort in-flight requests without pausing the scheduler (send `{}` to abort all, or `{"request_ids": [...]}`)
|
||||
|
||||
!!! note "Data Parallelism"
|
||||
When using data parallelism with vLLM's **internal load balancer** (i.e. `data_parallel_backend="ray"`), pause and resume are handled automatically across all DP ranks -- a single call is sufficient. When using an **external load balancer** (i.e. multiple independent vLLM instances behind a proxy), you must send pause and resume requests to **every** engine instance individually before and after the weight update.
|
||||
|
||||
@@ -191,6 +191,7 @@ The following endpoints **do not require authentication** even when `--api-key`
|
||||
- `/pause` - Pause generation (causes denial of service)
|
||||
- `/resume` - Resume generation
|
||||
- `/is_paused` - Check if generation is paused
|
||||
- `/abort_requests` - Abort in-flight requests (causes loss of in-flight work)
|
||||
- `/scale_elastic_ep` - Trigger scaling operations
|
||||
- `/is_scaling_elastic_ep` - Check if scaling is in progress
|
||||
- `/init_weight_transfer_engine` - Initialize weight transfer engine for RLHF
|
||||
|
||||
Reference in New Issue
Block a user