vLLM/docs/features/watermarking.md

Text watermarking

Text watermarking embeds a statistical signal in generated token choices. A detector that knows the generation parameters can test for this signal without access to the model weights.

The design and scope are discussed in RFC #53916.

Configuration

Enable watermarking by configuring an algorithm and secret key at engine startup:

vllm serve MODEL \
  --watermark-config '{"algorithm":"gumbel","key":42}'

Watermarking is disabled when --watermark-config is omitted. Gumbel-max (see gumbel) is the default algorithm within an enabled WatermarkConfig. When watermarking is configured, it is enabled for requests by default.

Requests can opt out without changing the engine-level algorithm or key:

from vllm import SamplingParams

sampling_params = SamplingParams(watermarking=False)

The OpenAI-compatible APIs accept the same watermarking: false request field. Deployments that require watermarking must restrict this field to trusted callers, or strip and validate it at the ingress boundary, so untrusted clients cannot opt out.

context_width controls how many prior output tokens seed each watermark decision and defaults to 4. Larger values make the watermark less robust to edits because an insertion, deletion, or substitution changes more subsequent contexts. Values above 16 are allowed but emit a warning.

Architecture

WatermarkConfig selects an algorithm and PRF. Model Runner V2 constructs the corresponding Watermarker, and GPUWatermarkSampler invokes it for the final stochastic token selection after temperature, min-p, top-k, and top-p are applied. A watermarker can either select a token directly or transform logits and delegate to vLLM's random sampler.

Detection is separate from generation. vLLM provides detector primitives for the reference algorithms. WatermarkDetector consumes token IDs, so callers remain responsible for using the tokenizer and watermark profile that match generation.

Algorithms

Gumbel-max

Gumbel-max derives a deterministic pseudorandom value from the key, prior generated-token context, and every candidate token, then uses the resulting Gumbel noise for categorical sampling. See Aaronson's original presentation.

Gumbel-max requires stochastic sampling. Greedy requests (temperature=0) bypass watermarking and emit a warning once per worker.

Single-key Gumbel-max watermarking may increase degenerate generations. In particular, reusing the same keyed random vector when a context repeats can reinforce repetition loops that continue until the maximum token limit.

SynthID-Text

SynthID-Text is planned but not currently implemented.

Pseudorandom functions

A watermark PRF turns the secret key, token context, and candidate token into reproducible random values. Generation and detection must produce identical values across devices and releases. The values should also be uniform and independent enough for the sampling algorithm and detector statistics. PRF selection is an advanced compatibility and performance setting; most users should keep the default.

Watermarked generation currently supports the philox PRF:

• philox is based on the counter-based Philox4x32-10 generator from the Random123 paper. It is parallel, vectorizes on accelerators, and avoids CPU transfers, but is not a cryptographic PRF and does not provide key-recovery or forgery resistance. vLLM versions its input mapping and provides compatibility vectors so generation and detection remain interoperable.

Detection

The detector primitives operate on token IDs and do not require model weights:

from transformers import AutoTokenizer

from vllm.v1.watermarking import GumbelWatermarkDetector

tokenizer = AutoTokenizer.from_pretrained(MODEL)
token_ids = tokenizer.encode(text, add_special_tokens=False)
result = GumbelWatermarkDetector(key=42, prf="philox").detect(token_ids)
print(result.p_value, result.is_watermarked)

The tokenizer, algorithm, PRF, key, and context width must match generation. Gumbel-max detection scores repeated contexts once by default so identical PRF random vectors are not treated as independent evidence. Keep deduplicate_contexts=True unless the detector's calibration has been adjusted for correlated scores.

The reported p-value is calibrated under the assumption that scored PRF inputs are independent. A deployment uses one fixed key, so repeated structures across documents reuse the same PRF values and can make the realized false-positive rate key-dependent even when contexts are deduplicated within each document. Measure the false-positive rate on representative unwatermarked traffic with the deployed key before relying on is_watermarked for decisions.

A minimal HTTP detector is available in examples/basic/online_serving/watermark_detection_server.py:

python examples/basic/online_serving/watermark_detection_server.py \
  --tokenizer MODEL --key 42 --prf philox

curl http://localhost:8000/detect \
  -H 'Content-Type: application/json' \
  -d '{"text":"Text to inspect"}'

Scores and p-values expose information about the per-token watermark signal. Repeated queries can use this information to construct text that imitates watermarked output or to modify watermarked text so it is no longer detected.

Limitations

• Watermarking is currently available only with Model Runner V2.
• Gumbel-max cannot be configured with speculative decoding.
• Beam search expands candidates from model log probabilities and does not apply Gumbel-max watermarking.
• Models that replace the vLLM sampler with a custom sampler cannot use configured watermarking.