[Rust Frontend] Expose gRPC port in Python vllm serve (#59659)

Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
Alec
2026-10-02 08:14:37 +00:00
committed by GitHub
co-authored by Bugen Zhao Claude Opus 5.5
parent b558f160a2
commit 9d2a6f52b2
11 changed files with 320 additions and 64 deletions
+21
View File
@@ -43,6 +43,27 @@ For example:
VLLM_USE_RUST_FRONTEND=1 vllm serve Qwen/Qwen3-0.6B
```
To also expose the Rust frontend's gRPC inference and control services, add
`--grpc-port`. The HTTP listener remains on `--port`:
```bash
VLLM_USE_RUST_FRONTEND=1 vllm serve Qwen/Qwen3-0.6B \
--host 127.0.0.1 --port 8000 --grpc-port 50051
```
The gRPC listener binds on `--host` like HTTP, or on `127.0.0.1` when HTTP uses
`--uds`. When HTTP uses TCP, its port and the gRPC port must differ unless
requesting an automatically assigned port with `0`.
This also works with Python-supervised hybrid data parallelism. It requires a
frontend process and cannot be combined with `--headless` or
`--data-parallel-multi-port-external-lb`.
The separate `vllm serve MODEL --grpc --port 50051` command launches the Python
SMG `VllmEngine` gRPC server in place of HTTP. The Rust listener exposes the
`vllm.Inference` and `vllm.Control` services alongside HTTP. These are different
gRPC APIs; `--grpc` and `--grpc-port` are mutually exclusive.
### RL weight synchronization
With `VLLM_SERVER_DEV_MODE=1`, the Rust frontend supports the HTTP weight-transfer