[Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216)

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
This commit is contained in:
Matt Mastracci
2026-09-24 13:15:08 -07:00
committed by GitHub
co-authored by Claude Fable 5.1 Lucas Wilkinson mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Misha Goin
parent 92e9c6bf13
commit ab3de6edf2
6 changed files with 262 additions and 6 deletions
@@ -11,8 +11,10 @@ left as noise, one denoise step gives a distribution over each slot. These
| `diffusion_pinned` | `list[int]` of canvas positions | held at their seed value on every denoise step, so a read past one step keeps its template |
| `diffusion_max_steps` | `int` | denoise steps before the canvas is emitted |
| `diffusion_read_only` | `bool` | emit the argmax canvas as soon as the cap is reached, end the request there, and return temperature-1 logprobs at every position |
| `diffusion_constrained` | `bool` | run the unembedding, sampler and self-conditioning over the request's `logprob_token_ids` only. Logprobs are normalized over that set. A step uses this only when every read in it has the same set |
`structured_server.py` turns a question schema into those fields. It serves
`structured_server.py` turns a question schema into those fields, with
`diffusion_constrained` on for every read (`--no-constrained` turns it off). It serves
`/v1/chat/completions`: the system message is the schema, the user message
is the state JSON, and the reply content is one distribution per question
with a standard error over a few noise draws.