[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
This commit is contained in:
Martin Hickey
2026-10-03 19:38:25 +08:00
committed by GitHub
parent 5f30fc7031
commit bc21cba967
18 changed files with 1356 additions and 169 deletions
+4 -2
View File
@@ -125,8 +125,10 @@ Scale-out APIs are disabled by default on `vllm serve`. Set `--enable-scale-out`
### Tokens IN <> Tokens OUT APIs
- `/inference/v1/generate` - Generate completions
- `/abort_requests` - Abort in-flight requests (only when `--tokens-only` is also set)
- [Generate API](token_in_token_out.md) (`/inference/v1/generate`)
- Generate completions from token IDs, optionally with detokenized text (`output_mode`)
- `/inference/v1/abort_requests` - Abort in-flight requests (registered wherever `/inference/v1/generate` is)
- `/abort_requests` - Unauthenticated alias of `/inference/v1/abort_requests` (`--tokens-only` only)
### Renderer APIs