[Rust Frontend][gRPC] Add RL lifecycle control (#51316)
Signed-off-by: Connor Carpenter <connorc@nvidia.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
co-authored by
Nick Hill
parent
3e3ceb1961
commit
f473870ecf
@@ -345,7 +345,7 @@ vLLM supports loading out-of-tree HTTP routes via the `vllm.endpoint_plugins` en
|
||||
|
||||
## gRPC Interface
|
||||
|
||||
vLLM provides an optional gRPC Generate service on a separate TCP port, enabled via the `--grpc-port` flag. When not specified, no gRPC server is started. The gRPC listener binds to the same host address as the HTTP server.
|
||||
vLLM provides optional gRPC `Inference` and `Control` services on a separate TCP port, enabled via the `--grpc-port` flag. When not specified, no gRPC server is started. The gRPC listener binds to the same host address as the HTTP server.
|
||||
|
||||
**Warning:** The gRPC interface is **insecure by default** — it does not implement authentication, authorization, or encryption. It should be considered a private, internal interface intended for use only between co-located services within a trusted network. Do not expose the gRPC port to the public internet or untrusted clients. If you enable the gRPC interface, protect it via network-level access controls such as firewall rules, network segmentation, or deployment on an isolated private network.
|
||||
|
||||
@@ -354,8 +354,9 @@ vLLM provides an optional gRPC Generate service on a separate TCP port, enabled
|
||||
An attacker who can reach the gRPC port can:
|
||||
|
||||
1. **Run arbitrary inference** via the `Generate` and `GenerateStream` RPCs without any credentials
|
||||
2. **Consume GPU and compute resources** by submitting unbounded generation requests
|
||||
3. **Cause Denial of Service** by exploiting bugs in the gRPC interface that can crash vLLM.
|
||||
2. **Mutate engine state** by pausing generation, sleeping the engine, or initiating configured RL weight updates through the `Control` service
|
||||
3. **Consume GPU and compute resources** by submitting unbounded generation requests
|
||||
4. **Cause Denial of Service** by exploiting bugs in the gRPC interface that can crash vLLM.
|
||||
|
||||
### Recommendations
|
||||
|
||||
|
||||
Reference in New Issue
Block a user