[UX] Speedup DeepGEMM warmup with heuristics (#25619)
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Michael Goin <mgoin64@gmail.com> Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
This commit is contained in:
co-authored by
gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Varun Sundar Rabindranath
parent
10214b6935
commit
0d21b9b51e
@@ -29,7 +29,7 @@ def kernel_warmup(worker: "Worker"):
|
||||
do_deep_gemm_warmup = (
|
||||
envs.VLLM_USE_DEEP_GEMM
|
||||
and is_deep_gemm_supported()
|
||||
and not envs.VLLM_SKIP_DEEP_GEMM_WARMUP
|
||||
and envs.VLLM_DEEP_GEMM_WARMUP != "skip"
|
||||
)
|
||||
if do_deep_gemm_warmup:
|
||||
model = worker.get_model()
|
||||
|
||||
Reference in New Issue
Block a user