[Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216)
Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Misha Goin <mgoin64@gmail.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
Lucas Wilkinson
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Misha Goin
parent
92e9c6bf13
commit
ab3de6edf2
@@ -11,8 +11,10 @@ left as noise, one denoise step gives a distribution over each slot. These
|
||||
| `diffusion_pinned` | `list[int]` of canvas positions | held at their seed value on every denoise step, so a read past one step keeps its template |
|
||||
| `diffusion_max_steps` | `int` | denoise steps before the canvas is emitted |
|
||||
| `diffusion_read_only` | `bool` | emit the argmax canvas as soon as the cap is reached, end the request there, and return temperature-1 logprobs at every position |
|
||||
| `diffusion_constrained` | `bool` | run the unembedding, sampler and self-conditioning over the request's `logprob_token_ids` only. Logprobs are normalized over that set. A step uses this only when every read in it has the same set |
|
||||
|
||||
`structured_server.py` turns a question schema into those fields. It serves
|
||||
`structured_server.py` turns a question schema into those fields, with
|
||||
`diffusion_constrained` on for every read (`--no-constrained` turns it off). It serves
|
||||
`/v1/chat/completions`: the system message is the schema, the user message
|
||||
is the state JSON, and the reply content is one distribution per question
|
||||
with a standard error over a few noise draws.
|
||||
|
||||
Reference in New Issue
Block a user