[Model Runner V2] Support randomized dummy inputs (#58411)
Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
co-authored by
Robert Shaw
Claude Opus 5.5
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Nick Hill
parent
5addde2a0a
commit
e5e38ba9b7
@@ -721,7 +721,12 @@ class ElasticEPScalingExecutor:
|
||||
self._suppress_eplb(),
|
||||
serving_state,
|
||||
):
|
||||
runner._dummy_run(runner.max_num_tokens, is_profile=True, skip_eplb=True)
|
||||
runner._dummy_run(
|
||||
runner.max_num_tokens,
|
||||
is_profile=True,
|
||||
skip_eplb=True,
|
||||
randomize_inputs=self.worker.randomize_dummy_inputs,
|
||||
)
|
||||
self.worker.compile_or_warm_up_model()
|
||||
|
||||
lock_workspace()
|
||||
|
||||
Reference in New Issue
Block a user