[MM][CG] Support ViT CG for Qwen2-VL (#41736)

Signed-off-by: John Calderon <jcalderon@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
John Calderon
2026-05-14 01:52:35 +08:00
committed by GitHub
co-authored by Roger Wang
parent 2f821faeae
commit b3c69595a6
4 changed files with 315 additions and 21 deletions
+2 -1
View File
@@ -85,13 +85,14 @@ Models opt-in to encoder CUDA Graphs by implementing the [SupportsEncoderCudaGra
| Architecture | Models | CG for Image | CG for Video |
| ------------ | ------ | ------------ | ------------ |
| `Qwen2VLForConditionalGeneration` | `Qwen2-VL` | ✅︎ | ✅︎ |
| `Qwen2_5_VLForConditionalGeneration` | `Qwen2.5-VL` | ✅︎ | ✅︎ |
| `Qwen3VLForConditionalGeneration` | `Qwen3-VL` | ✅︎ | ✅︎ |
| `Qwen3_5ForConditionalGeneration` | `Qwen3.5` | ✅︎ | ✅︎ |
!!! note
Encoder CUDA Graphs have currently been tested with `--mm-encoder-attn-backend=FLASH_ATTN` and `--mm-encoder-attn-backend=FLASHINFER` on Blackwell GPUs.
For Qwen2.5-VL only FA2 and FA3 has been tested.
For Qwen2-VL and Qwen2.5-VL only FA2 and FA3 has been tested.
## Configuration