[Feature] Parse request priority from HTTP header (#51089)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
This commit is contained in:
Chauncey
2026-08-05 21:10:29 -07:00
committed by GitHub
parent 2e09247c2d
commit 22170354d8
8 changed files with 81 additions and 8 deletions
@@ -83,8 +83,8 @@ completion = client.chat.completions.create(
## Extra HTTP Headers
Only `X-Request-Id` HTTP request header is supported for now. It can be enabled
with `--enable-request-id-headers`.
The `X-Request-Id` HTTP request header can be enabled with
`--enable-request-id-headers`.
??? code
@@ -110,6 +110,19 @@ with `--enable-request-id-headers`.
print(completion._request_id)
```
The Completions, Chat Completions, and Responses APIs also support the
`X-Vllm-Priority` request header. Its value must be an integer and overrides
the `priority` value in the JSON request body. Non-zero priorities require the
server to use priority scheduling.
```python
completion = client.chat.completions.create(
model="NousResearch/Meta-Llama-3-8B-Instruct",
messages=[{"role": "user", "content": "Hello!"}],
extra_headers={"X-Vllm-Priority": "-10"},
)
```
## API Reference
### Completions API