[Frontend] structured decisions: noul and score question types (#60696)

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: DeepSeek <noreply@deepseek.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
This commit is contained in:
Matt Mastracci
2026-10-09 04:48:16 +00:00
committed by GitHub
co-authored by DeepSeek Claude Opus 5.5 Misha Goin
parent 88a419649e
commit 8cbd5d0300
6 changed files with 270 additions and 27 deletions
@@ -11,8 +11,8 @@ log probabilities.
## How it works
1. Each question is one read. The user message is the state, followed by the
question and its options, labeled `A`, `B`, `C`, ... in order, and
"Answer with the letter of one option only."
question and its options with the type's labels, and an instruction to
answer with one label.
2. The read is the reply's first token. Each label must be one token there,
or the request returns 400.
3. The read returns the logprobs of that question's label tokens. Softmax over
@@ -31,8 +31,11 @@ question prefills only its own text, then reads one token.
| type | criteria | answer |
| --- | --- | --- |
| `choice` | map of option name to a description or `null` | `choice`, `probabilities` by option name, `confidence` |
| `noul` | `null`, or an object with optional `true` and `false` descriptions | `noul` (probability of yes), `probabilities`, `confidence` |
| `score` | ordered list of level names | `score` (expected 0-indexed level), `legend`, `probabilities` by level index, `confidence` |
A choice has at least one option. A question id is a non-empty string.
A choice has at least one option, a score at least two levels. A question id
is a non-empty string.
## Example
@@ -51,6 +54,16 @@ curl -s http://localhost:8000/v1/systemone \
"shipping": "deliveries",
"security": "account access"
}
},
"urgent": {
"type": "noul",
"instructions": "Does the customer need a reply within the hour?",
"criteria": {"true": "the account is locked or money is moving"}
},
"tone": {
"type": "score",
"instructions": "How upset is the customer?",
"criteria": ["calm", "annoyed", "furious"]
}
},
"chat_template_kwargs": {"enable_thinking": false}
@@ -71,11 +84,26 @@ Response, with probabilities that depend on the model:
"choice": "billing",
"probabilities": {"billing": 0.97, "shipping": 0.01, "security": 0.02},
"confidence": 0.90
},
"urgent": {
"type": "noul",
"noul": 0.02,
"probabilities": {"yes": 0.02, "no": 0.98},
"confidence": 0.95
},
"tone": {
"type": "score",
"score": 0.6,
"legend": {"0": "calm", "1": "annoyed", "2": "furious"},
"probabilities": {"0": 0.7, "1": 0.2, "2": 0.1},
"confidence": 0.65
}
},
"usage": {"input_tokens": 142, "output_tokens": 1},
"usage": {"input_tokens": 142, "output_tokens": 3},
"diagnostics": {
"team": {"label_mass": 0.93, "argmax_is_label": true}
"team": {"label_mass": 0.93, "argmax_is_label": true},
"urgent": {"label_mass": 0.98, "argmax_is_label": true},
"tone": {"label_mass": 0.91, "argmax_is_label": true}
}
}
```
@@ -87,8 +115,9 @@ whole vocabulary: its share of the labels times `label_mass`.
## Labels
Options are labeled `A` to `Z` in the order the request lists them, so a
`choice` has at most 26 options.
Labels follow the type: a `choice` labels its options `A` to `Z` in the order
the request lists them, a `noul` labels its options `yes` and `no`, and a
`score` labels its levels `0` to `9`, so a level's label is its score.
## Limits
@@ -96,6 +125,7 @@ Options are labeled `A` to `Z` in the order the request lists them, so a
| --- | --- |
| questions per request | 64 |
| options per `choice` | 26 |
| levels per `score` | 10 |
## Request fields