[warmup] overlap renderer warmup and engine core initialization (#52764)

Signed-off-by: Andy Xie <andy.xning@gmail.com>
This commit is contained in:
Ning Xie
2026-08-27 11:42:08 +08:00
committed by GitHub
parent f8d38fbc87
commit ca90b9e7d4
4 changed files with 345 additions and 24 deletions
+17
View File
@@ -93,6 +93,19 @@ class LLMEngine:
# Convert EngineInput --> EngineCoreRequest.
self.input_processor = InputProcessor(self.vllm_config, renderer)
# Launch the multimodal warmup in the background so it overlaps
# engine-core initialization (model loading, started below). Must be
# after InputProcessor: its MultiModalBudget runs a synchronous MM
# processing pass (get_dummy_mm_inputs -> processor.apply), and the HF
# processor's Numba parallel kernels (e.g. Kimi-K2.5 vision fused) are
# not thread-safe — launching the warmup thread any earlier would run
# it concurrently with that pass and abort. The MM warmup is
# frontend-only (sends no engine_core requests) and only needs the
# renderer. Joined by reset_mm_cache / warmup / shutdown so the
# mm_processor_cache is never touched concurrently by the warmup and
# the serving path.
renderer.start_mm_warmup_in_background()
# Converts EngineCoreOutputs --> RequestOutput.
self.output_processor = OutputProcessor(
renderer.tokenizer,
@@ -343,6 +356,10 @@ class LLMEngine:
self.engine_core.profile(False)
def reset_mm_cache(self):
# Join any background MM warmup before clearing: the mm_processor_cache
# is not designed for concurrent access, so the warmup's apply/clear
# must have finished before we clear here.
self.renderer._join_mm_warmup()
self.renderer.clear_mm_cache()
self.engine_core.reset_mm_cache()