[MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference (#40445)

Signed-off-by: shen-shanshan <467638484@qq.com>
This commit is contained in:
Shanshan Shen
2026-04-21 14:47:53 +00:00
committed by GitHub
parent b2a5518679
commit 936e0b79aa
6 changed files with 46 additions and 15 deletions
+6
View File
@@ -1531,6 +1531,12 @@ class SupportsEncoderCudaGraph(Protocol):
"""Return the modality of the inputs."""
...
def get_max_frames_per_video(
self,
) -> int:
"""Return model-specific max frames per video."""
...
def get_encoder_cudagraph_budget_range(
self,
vllm_config: "VllmConfig",