[watermarking] add context deduplication support to speculative decoding (#56807)

Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
Raphaël Rialland
2026-09-30 12:22:10 -04:00
committed by GitHub
co-authored by Simon Veitner Lucas Wilkinson
parent 73a5831127
commit 0f8b398158
17 changed files with 1379 additions and 88 deletions
+14 -15
View File
@@ -77,30 +77,17 @@ watermarked. The watermark signal is diluted in proportion to the share of
output tokens supplied by accepted drafts; rejected drafts do not dilute it
because their recovery tokens are watermarked.
Speculative-decoding token paths do not currently support generation-side
context deduplication. The configured `deduplicate_contexts` policy is not
applied to accepted drafts, rejection-recovery tokens, or bonus tokens.
For `dual_key_gumbel`, `alpha` has no effect under speculative decoding. The
speculative protocol selects the key for each token instead.
## Algorithms
### Gumbel-max
Gumbel-max derives a deterministic pseudorandom value from the key, prior token
context, and every candidate token, then uses the resulting Gumbel noise for
categorical sampling. See
[Aaronson's original presentation](https://simons.berkeley.edu/sites/default/files/2024-10/LLM24-2%20Slides%20-%20Scott%20Aaronson.pdf).
Gumbel-max requires stochastic sampling. Greedy requests (`temperature=0`)
bypass watermarking and emit a warning once per worker.
## Context deduplication
When a token context is repeated, generation can use ordinary sampling for that
occurrence. Reusing the repeated context leads to a bias over the sequence, as
certain token choices would be correlated. Ordinary sampling at these positions
allows for single-sequence non-distortion (see section G.3 of the
[SynthID-Text supplementary materials](https://media.springernature.com/original/springer-static/esm/art%3A10.1038%2Fs41586-024-08025-4/MediaObjects/41586_2024_8025_MOESM1_ESM.pdf)).
This also applies to drafter watermarking during speculative decoding.
The detector independently deduplicates contexts so repeated keyed random
vectors are not treated as independent evidence, meaning that context
deduplication at generation time does not reduce the watermarking signal, unless
@@ -143,6 +130,18 @@ vllm serve MODEL \
'{"algorithm":"gumbel","key":42,"deduplicate_contexts":"all"}'
```
## Algorithms
### Gumbel-max
Gumbel-max derives a deterministic pseudorandom value from the key, prior token
context, and every candidate token, then uses the resulting Gumbel noise for
categorical sampling. See
[Aaronson's original presentation](https://simons.berkeley.edu/sites/default/files/2024-10/LLM24-2%20Slides%20-%20Scott%20Aaronson.pdf).
Gumbel-max requires stochastic sampling. Greedy requests (`temperature=0`)
bypass watermarking and emit a warning once per worker.
### Dual-key Gumbel-max
Dual-key Gumbel-max derives independent keys A and B from one configured master