[Rust Frontend][gRPC] Add RL lifecycle control (#51316)
Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
co-authored by
Nick Hill
parent
3e3ceb1961
commit
f473870ecf
@@ -158,6 +158,8 @@ When running vLLM as an HTTP server, the following endpoints are available for w
|
||||
!!! note
|
||||
The HTTP weight transfer endpoints require `VLLM_SERVER_DEV_MODE=1` to be set.
|
||||
|
||||
The Rust frontend's optional gRPC `Control` service exposes the same pause, sleep, weight-transfer, and weight-version lifecycle for trusted sidecars. The `ServerInfo.rl_capabilities` response reports whether weight transfer and sleep mode were configured. Backend-specific `init_info` and `update_info` remain JSON metadata; model tensors continue to move over the configured NCCL, IPC, or sparse-NCCL transport.
|
||||
|
||||
## Extending the System
|
||||
|
||||
Every piece of the system is replaceable: the weights you send (`WeightSource`),
|
||||
|
||||
Reference in New Issue
Block a user