[Feat] Add text-only OpenAI Decisions API MVP (#60465)

Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
This commit is contained in:
Francisco Javier Arceo
2026-10-14 07:51:08 +08:00
committed by GitHub
co-authored by Codex Misha Goin
parent ad8d39cbab
commit c923e7769b
23 changed files with 1209 additions and 97 deletions
+2
View File
@@ -16,6 +16,8 @@ We currently support the following OpenAI APIs:
- [Chat Completions batch API](./openai_compatible_server.md#chat-api) (`/v1/chat/completions/batch`)
- [Responses API](./openai_compatible_server.md#responses-api) (`/v1/responses`, `/v1/responses/{response_id}`, `/v1/responses/{response_id}/cancel`)
- Only applicable to [text generation models](../../models/generative_models.md).
- [Decisions API](decisions.md) (`/v1/decisions`)
- Supports text-only predicate, choice, and score questions on supported generation models.
- [Embeddings API](../../models/pooling_models/embed.md#openai-compatible-embeddings-api) (`/v1/embeddings`)
- Only applicable to [embedding models](../../models/pooling_models/embed.md).
- [Transcriptions API](./speech_to_text.md#transcriptions-api) (`/v1/audio/transcriptions`)
+70
View File
@@ -0,0 +1,70 @@
# Decisions API
vLLM provides a text-only implementation of OpenAI's
[`POST /v1/decisions`](https://developers.openai.com/api/docs/guides/decisions)
API. Start a supported model:
```bash
vllm serve Qwen/Qwen3-0.6B
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
decision = client.decisions.create(
model="Qwen/Qwen3-0.6B",
input="The package arrived with a broken screen.",
questions=[
{
"type": "predicate",
"name": "damaged",
"instructions": "Does the customer report a damaged item?",
},
{
"type": "choice",
"name": "department",
"instructions": "Which department should handle this complaint?",
"choices": [{"value": "returns"}, {"value": "billing"}],
},
{
"type": "score",
"instructions": "How severe is the damage?",
"levels": [{"label": "cosmetic"}, {"label": "broken"}],
},
],
)
print(decision.answers)
```
Use OpenAI Python SDK 3.26.0 or newer for `client.decisions.create`.
`input` accepts a string or an array of user messages with string content or
`input_text` parts. Answers retain question order and echo `name`, including
`null` for unnamed questions. Choice values preserve strings and booleans.
Scores are probability-weighted averages of zero-based level indices.
The request contract follows the OpenAI OpenAPI schemas retrieved on
October 7, 2026. This MVP supports 1–64 questions, 2–26 choices, and 2–10 score
levels. Upstream permits 200 questions and 255 choices; this backend returns
a 400 when either model limit is exceeded.
The optional `safety_identifier` is caller metadata, available in request
body logging with `--enable-log-requests` and DEBUG logging; it does not
authenticate callers.
## Implementation and limits
This MVP uses the same label-reading backend as
[structured decisions](structured_decisions.md). It supports generation
architectures with a registered read strategy, including Qwen, DiffusionGemma,
and Liquid d1; unsupported models return 501. Each question reads a model's
label tokens. Usage reports the actual prompt and output tokens across these
reads, including prefix-cache counters.
Probabilities are normalized over the supplied options. `confidence` is the
winning label's probability over the full vocabulary; it is not calibrated
to OpenAI's models. Thinking is disabled for these reads.
Image inputs, other message roles, tools, streaming, and per-question refusal
scoring are outside this MVP. Unsupported request fields and input types are
rejected. The `/v1/systemone` endpoint retains its existing request format and
is available by default.