[Benchmark] Add sweep warmup and failure recovery (#57305)

Signed-off-by: louie-tsai <louie.tsai@intel.com>
This commit is contained in:
Louie Tsai
2026-09-30 11:16:41 +08:00
committed by GitHub
parent 42d9f8ccdb
commit 1b77cc3d67
4 changed files with 299 additions and 34 deletions
+6
View File
@@ -88,6 +88,12 @@ vllm bench sweep serve \
By default, each parameter combination is benchmarked 3 times to make the results more reliable. You can adjust the number of runs by setting `--num-runs`.
You can optionally run an unmeasured warmup before each parameter combination by setting `--warmup-num-prompts`. The warmup result is saved as `warmup.json` for inspection, but is excluded from the measured summary.
You can use `--continue-on-error` to record a failed warmup, benchmark run, or server configuration and continue processing the remaining sweep combinations. Failure details are saved in `.failure.json` files.
Both options are also supported by `vllm bench sweep serve_workload`.
!!! important
If both `--serve-params` and `--bench-params` are passed, the script will iterate over the Cartesian product between them.
You can use `--dry-run` to preview the commands to be run.