[Rust Frontend] Add support for TLS in render server (#54999)

Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Richard <noreply@moonshot.cn>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
Wen Zhou
2026-09-03 21:56:44 +00:00
committed by GitHub
co-authored by Bugen Zhao Richard
parent 6fdee17f84
commit c7e6e36fa9
5 changed files with 215 additions and 64 deletions
+21
View File
@@ -90,6 +90,27 @@ It exposes `/v1/chat/completions/render` and `/v1/completions/render`. Only
tokenizer and model configuration files are loaded; model weights, PyTorch,
and vLLM kernels are not required.
To serve HTTPS, pass `--ssl-certfile`. If the certificate and private key are
in separate files, pass `--ssl-keyfile` for the key; otherwise the key is read from
the certificate file. For mTLS, pass `--ssl-ca-certs` with a CA
bundle and set `--ssl-cert-reqs` to `1` (optional) or `2` (required) to
verify client certificates. Use `--ssl-ciphers` to override the default
OpenSSL cipher list.
```bash
# Combined PEM (cert + key in one file):
vllm-rs \
render Qwen/Qwen3-32B \
--host 127.0.0.1 --max-model-len 32768 \
--ssl-certfile /path/to/combined.pem
# Separate cert and key files:
vllm-rs \
render Qwen/Qwen3-32B \
--host 127.0.0.1 --max-model-len 32768 \
--ssl-certfile /path/to/cert.pem --ssl-keyfile /path/to/key.pem
```
The render endpoints return the public token-in `GenerateRequest` consumed by
the Rust `/inference/v1/generate` endpoint. A chat render response, or one item
from a completion render response, can be submitted to that endpoint without