[watermark] compatibility validation (#56801)

Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
This commit is contained in:
Raphaël Rialland
2026-10-05 14:37:13 -04:00
committed by GitHub
co-authored by Simon Veitner
parent 88f3c601b3
commit c32495d58f
45 changed files with 1670 additions and 101 deletions
+42 -11
View File
@@ -18,8 +18,16 @@ vllm serve MODEL \
```
Watermarking is disabled when `--watermark-config` is omitted. Gumbel-max (see
`gumbel`) is the default algorithm within an enabled `WatermarkConfig`. When
watermarking is configured, it is enabled for requests by default.
`gumbel`) is the default algorithm within an enabled `WatermarkConfig`.
The request-level `watermarking` option has two effective states:
- Omitted (`None` in Python) or `true` requests watermarking.
- `false` explicitly opts out.
When a request asks for watermarking but the engine or generation mode cannot
apply it, vLLM emits a server-side warning once and runs the request without a
watermark.
Requests can opt out without changing the engine-level algorithm or key:
@@ -30,14 +38,12 @@ sampling_params = SamplingParams(watermarking=False)
```
The Python OpenAI-compatible APIs and the Rust chat/completions APIs accept the
same `watermarking: false` request field. The Rust token API accepts it in
`sampling_params`, and the Rust gRPC `GenerateRequest` accepts
the optional `watermarking` field. Omitting the field keeps watermarking enabled
when the engine has a watermark configuration.
same optional `watermarking` request field. The Rust token API accepts it in
`sampling_params`, and the Rust gRPC `GenerateRequest` accepts it directly.
Omitting the field requests watermarking by default.
Deployments that require watermarking must restrict this field to trusted
callers, or strip and validate it at the ingress boundary, so untrusted clients
cannot opt out.
Deployments that require watermarking must restrict explicit opt-out to trusted
callers, or strip and validate the field at the ingress boundary.
`context_width` controls how many prior tokens seed each watermark decision
and defaults to 4. Larger values make the watermark less robust to
@@ -63,6 +69,31 @@ the reference algorithms. `WatermarkDetector` consumes token IDs, so callers
remain responsible for using the tokenizer and watermark profile that match
generation.
## Interactions with generation features
- Greedy decoding (`temperature=0`) cannot apply a Gumbel watermark. The first
such request emits a server-side warning; it and subsequent greedy requests
use ordinary greedy sampling without a watermark. This includes
transcription, translation, and realtime transcription, which default to
greedy decoding. Transcription and translation can use a nonzero temperature
to apply the configured watermark.
- Beam search cannot apply a watermark. The first watermarked beam request emits
a server-side warning and runs without watermarking.
- Trace replay replaces the sampled token and therefore cannot apply a
watermark. The first watermarked trace replay request emits a server-side
warning and runs without watermarking.
- Parallel sampling (`n > 1`) is supported, but plain Gumbel candidates can be
identical while their contexts remain identical.
- Structured outputs and tool grammars are applied before watermark sampling.
Restrictive constraints can reduce the statistical evidence available to a
detector.
- Automatic language detection and generative scoring use unwatermarked
auxiliary generations. Language-detection tokens are not part of the returned
transcription, and generative scoring returns scores rather than generated
text.
- Speculative decoding has additional requirements and partial-watermarking
modes described in [Speculative decoding](#speculative-decoding).
## Speculative decoding
Watermarking requires speculative decoding to use probabilistic draft sampling,
@@ -245,7 +276,7 @@ watermarked output or to modify watermarked text so it is no longer detected.
- Watermarking is currently available only with Model Runner V2.
- Not all watermarking algorithms have native speculative-decoding support.
- Beam search expands candidates from model log probabilities and does not apply
Gumbel-max watermarking.
- Models that replace the vLLM sampler with a custom sampler cannot use
configured watermarking.
- Global custom logits processors are unavailable because Model Runner V2 does
not support them.