[Benchmark] Add probe requests to vllm bench serve (#49611)

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
This commit is contained in:
Guan-Ming Chiu
2026-08-01 22:10:56 +08:00
committed by GitHub
parent ab06486175
commit 63e78ce365
3 changed files with 106 additions and 0 deletions
+25
View File
@@ -622,6 +622,31 @@ The following arguments can be used to control the ramp-up:
- `--ramp-up-start-rps`: The request rate at the beginning of the benchmark.
- `--ramp-up-end-rps`: The request rate at the end of the benchmark.
#### Probe Requests
The benchmark tool also supports sending probe requests alongside the main
workload. This can be useful for measuring how the main workload affects
unrelated traffic sharing the server, e.g. a few requests with large images
stalling a concurrent lightweight request while their multimodal preprocessing
occupies the frontend.
Setting `--probe-request-rate` to a positive value sends single-token text-only
probe requests at that rate (requests per second) alongside the main workload.
Probes bypass `--max-concurrency` and their latency is reported separately, so
the probe percentiles directly measure the interference that the main workload
inflicts on unrelated requests.
```bash
vllm bench serve \
--model Qwen/Qwen2.5-VL-3B-Instruct \
--backend openai-chat \
--endpoint /v1/chat/completions \
--dataset-name random-mm \
--random-mm-bucket-config '{(2048, 2048, 1): 1.0}' \
--request-rate 4 \
--probe-request-rate 20
```
#### Load Pattern Configuration
vLLM's benchmark serving script provides sophisticated load pattern simulation capabilities through three key parameters that control request generation and concurrency behavior: