[watermark] compatibility validation (#56801)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai> Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Simon Veitner <sveitner@redhat.com>
This commit is contained in:
co-authored by
Simon Veitner
parent
88f3c601b3
commit
c32495d58f
@@ -18,8 +18,16 @@ vllm serve MODEL \
|
||||
```
|
||||
|
||||
Watermarking is disabled when `--watermark-config` is omitted. Gumbel-max (see
|
||||
`gumbel`) is the default algorithm within an enabled `WatermarkConfig`. When
|
||||
watermarking is configured, it is enabled for requests by default.
|
||||
`gumbel`) is the default algorithm within an enabled `WatermarkConfig`.
|
||||
|
||||
The request-level `watermarking` option has two effective states:
|
||||
|
||||
- Omitted (`None` in Python) or `true` requests watermarking.
|
||||
- `false` explicitly opts out.
|
||||
|
||||
When a request asks for watermarking but the engine or generation mode cannot
|
||||
apply it, vLLM emits a server-side warning once and runs the request without a
|
||||
watermark.
|
||||
|
||||
Requests can opt out without changing the engine-level algorithm or key:
|
||||
|
||||
@@ -30,14 +38,12 @@ sampling_params = SamplingParams(watermarking=False)
|
||||
```
|
||||
|
||||
The Python OpenAI-compatible APIs and the Rust chat/completions APIs accept the
|
||||
same `watermarking: false` request field. The Rust token API accepts it in
|
||||
`sampling_params`, and the Rust gRPC `GenerateRequest` accepts
|
||||
the optional `watermarking` field. Omitting the field keeps watermarking enabled
|
||||
when the engine has a watermark configuration.
|
||||
same optional `watermarking` request field. The Rust token API accepts it in
|
||||
`sampling_params`, and the Rust gRPC `GenerateRequest` accepts it directly.
|
||||
Omitting the field requests watermarking by default.
|
||||
|
||||
Deployments that require watermarking must restrict this field to trusted
|
||||
callers, or strip and validate it at the ingress boundary, so untrusted clients
|
||||
cannot opt out.
|
||||
Deployments that require watermarking must restrict explicit opt-out to trusted
|
||||
callers, or strip and validate the field at the ingress boundary.
|
||||
|
||||
`context_width` controls how many prior tokens seed each watermark decision
|
||||
and defaults to 4. Larger values make the watermark less robust to
|
||||
@@ -63,6 +69,31 @@ the reference algorithms. `WatermarkDetector` consumes token IDs, so callers
|
||||
remain responsible for using the tokenizer and watermark profile that match
|
||||
generation.
|
||||
|
||||
## Interactions with generation features
|
||||
|
||||
- Greedy decoding (`temperature=0`) cannot apply a Gumbel watermark. The first
|
||||
such request emits a server-side warning; it and subsequent greedy requests
|
||||
use ordinary greedy sampling without a watermark. This includes
|
||||
transcription, translation, and realtime transcription, which default to
|
||||
greedy decoding. Transcription and translation can use a nonzero temperature
|
||||
to apply the configured watermark.
|
||||
- Beam search cannot apply a watermark. The first watermarked beam request emits
|
||||
a server-side warning and runs without watermarking.
|
||||
- Trace replay replaces the sampled token and therefore cannot apply a
|
||||
watermark. The first watermarked trace replay request emits a server-side
|
||||
warning and runs without watermarking.
|
||||
- Parallel sampling (`n > 1`) is supported, but plain Gumbel candidates can be
|
||||
identical while their contexts remain identical.
|
||||
- Structured outputs and tool grammars are applied before watermark sampling.
|
||||
Restrictive constraints can reduce the statistical evidence available to a
|
||||
detector.
|
||||
- Automatic language detection and generative scoring use unwatermarked
|
||||
auxiliary generations. Language-detection tokens are not part of the returned
|
||||
transcription, and generative scoring returns scores rather than generated
|
||||
text.
|
||||
- Speculative decoding has additional requirements and partial-watermarking
|
||||
modes described in [Speculative decoding](#speculative-decoding).
|
||||
|
||||
## Speculative decoding
|
||||
|
||||
Watermarking requires speculative decoding to use probabilistic draft sampling,
|
||||
@@ -245,7 +276,7 @@ watermarked output or to modify watermarked text so it is no longer detected.
|
||||
|
||||
- Watermarking is currently available only with Model Runner V2.
|
||||
- Not all watermarking algorithms have native speculative-decoding support.
|
||||
- Beam search expands candidates from model log probabilities and does not apply
|
||||
Gumbel-max watermarking.
|
||||
- Models that replace the vLLM sampler with a custom sampler cannot use
|
||||
configured watermarking.
|
||||
- Global custom logits processors are unavailable because Model Runner V2 does
|
||||
not support them.
|
||||
|
||||
Reference in New Issue
Block a user