[Metrics] Count finished requests by watermark status (#58715)

Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
simon-veitner-redhat
2026-10-12 17:21:22 +00:00
committed by GitHub
co-authored by Claude Lucas Wilkinson
parent 45bb710eca
commit a7b7bc1fbe
7 changed files with 203 additions and 1 deletions
+28
View File
@@ -56,6 +56,34 @@ watermarking algorithm that does not support it natively, at the cost of
weaker detectability. See
[Speculative decoding](#speculative-decoding).
## Monitoring
When `watermark_config` is set, the Prometheus counter
`vllm:watermark_requests_total` counts finished requests by `status`:
- `watermarked`: the request was sampled with the watermark switched on.
- `skipped`: the request asked for the watermark (omitted or `true`) but could
not use it, such as `temperature=0` and trace replay requests. A beam search
request that asks for the watermark runs without it and logs a warning. Its
beam steps count as `not_watermarked`, not `skipped`.
- `not_watermarked`: all other finished requests. This includes explicit
opt-outs, pooling requests, and the internal requests of beam search (one
per beam step), generative scoring and language detection.
Each finished request has exactly one status, so the sum over `status` equals
`vllm:request_success_total` on the same instance. Requests that the client
aborts are in neither counter. The counter does not exist without
`watermark_config`. With `n > 1`, each sequence counts once. In a
disaggregated prefill/decode deployment, both instances count a request: sum
over decode instances only. For streaming input, the first input chunk sets
the status.
The status is per request, not per token. A `watermarked` request can contain
tokens without the watermark: accepted draft tokens with
`allow_target_only_watermarking`, tokens whose context repeats
(`deduplicate_contexts`), the first `context_width` generated tokens with
`deduplicate_contexts: "all"`, and tokens that a grammar forces.
## Architecture
`WatermarkConfig` selects an algorithm and PRF. Model Runner V2 constructs the