[Benchmark] Add sweep warmup and failure recovery (#57305)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
This commit is contained in:
@@ -88,6 +88,12 @@ vllm bench sweep serve \
|
||||
|
||||
By default, each parameter combination is benchmarked 3 times to make the results more reliable. You can adjust the number of runs by setting `--num-runs`.
|
||||
|
||||
You can optionally run an unmeasured warmup before each parameter combination by setting `--warmup-num-prompts`. The warmup result is saved as `warmup.json` for inspection, but is excluded from the measured summary.
|
||||
|
||||
You can use `--continue-on-error` to record a failed warmup, benchmark run, or server configuration and continue processing the remaining sweep combinations. Failure details are saved in `.failure.json` files.
|
||||
|
||||
Both options are also supported by `vllm bench sweep serve_workload`.
|
||||
|
||||
!!! important
|
||||
If both `--serve-params` and `--bench-params` are passed, the script will iterate over the Cartesian product between them.
|
||||
You can use `--dry-run` to preview the commands to be run.
|
||||
|
||||
Reference in New Issue
Block a user