[MM][Perf][CG] Support ViT full cudagraphs for mllama4 (#40660)

Signed-off-by: allgather <all2allops@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
allgather
2026-06-11 22:17:55 -07:00
committed by GitHub
co-authored by Isotr0py
parent eb28452b10
commit 39dee1114a
5 changed files with 193 additions and 14 deletions
+9
View File
@@ -82,6 +82,7 @@ Models opt-in to encoder CUDA Graphs by implementing the [SupportsEncoderCudaGra
| Architecture | Models | CG for Image | CG for Video |
| ------------ | ------ | ------------ | ------------ |
| `Llama4ForConditionalGeneration` | `Llama 4` | ✅︎ | - |
| `InternVLChatModel` | `InternVL3.5`, `InternVL3`, `InternVL2.5`, `InternVL2` | ✅︎ | ✅︎ |
| `Qwen2VLForConditionalGeneration` | `Qwen2-VL` | ✅︎ | ✅︎ |
| `Qwen2_5_VLForConditionalGeneration` | `Qwen2.5-VL` | ✅︎ | ✅︎ |
@@ -114,6 +115,14 @@ vllm serve Qwen/Qwen3-VL-32B \
--compilation-config '{"cudagraph_mm_encoder": true}'
```
For `Llama 4` (image only):
```bash
vllm serve meta-llama/Llama-4-Scout-17B-16E-Instruct \
--limit-mm-per-prompt '{"image": 1}' \
--compilation-config '{"cudagraph_mm_encoder": true}'
```
With explicit budgets:
```bash