[Perf] Isolate MM preprocessing on its own executor (#49524)

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
This commit is contained in:
Guan-Ming Chiu
2026-07-26 08:04:17 +00:00
committed by GitHub
parent 0164022c90
commit 8d28b48d01
3 changed files with 31 additions and 20 deletions
+4 -3
View File
@@ -748,12 +748,13 @@ class ModelConfig:
if (
self.renderer_num_workers > 1
and self.multimodal_config.mm_processor_cache_gb > 0
and self.runner_type == "pooling"
):
raise ValueError(
"Cannot use --renderer-num-workers > 1 with the "
"multimodal processor cache enabled. The cache is "
"not thread-safe and does not support concurrent "
"renderer workers. Please set "
"multimodal processor cache enabled for pooling models. "
"Pooling preprocessing runs on the renderer workers, and "
"the cache is not thread-safe. Please set "
"--renderer-num-workers 1 (the default), or "
"disable the cache with --mm-processor-cache-gb 0."
)