[multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)

Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Brandon Pelfrey
2026-07-24 23:31:41 -07:00
committed by GitHub
co-authored by OpenAI Codex
parent 0ba2aa35a8
commit 94682b79f4
7 changed files with 189 additions and 23 deletions
+21 -4
View File
@@ -818,16 +818,18 @@ Full example: [examples/generate/multimodal/openai_chat_completion_client_for_mu
#### Video Decoding Backend
vLLM decodes video bytes into frames using a selectable decoding backend. Three
vLLM decodes video bytes into frames using a selectable decoding backend. Five
backends are supported:
- `opencv` (default): OpenCV-based decoder.
- `pyav`: PyAV decoder.
- `torchcodec`: TorchCodec (PyTorch-native) decoder.
- `pynvvideocodec`: NVIDIA NVDEC-based decoder.
- `deepstream`: NVIDIA DeepStream NVDEC-based decoder.
All three backends are ultimately backed by FFmpeg. `torchcodec` lets
you choose which FFmpeg version is used while `opencv` and `pyav` rely on
whichever FFmpeg build they were linked against.
The CPU backends are backed by FFmpeg. `torchcodec` lets you choose which FFmpeg
version is used while `opencv` and `pyav` rely on whichever FFmpeg build they
were linked against.
Select the backend by passing the `backend` parameter via `--media-io-kwargs`:
@@ -854,6 +856,21 @@ vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \
--media-io-kwargs '{"video": {"backend": "torchcodec", "seek_mode": "approximate", "num_ffmpeg_threads": 4}}'
```
**PyNvVideoCodec-specific parameters:**
- `hw_decoders`: Maximum number of concurrent hardware decoder slots retained
by each API server process. It must be a positive integer and defaults to `2`,
which is the recommended starting point for concurrent video workloads.
Because vLLM reserves GPU memory for these slots at startup, this value cannot
be overridden per request. Benchmark before increasing it because each
additional slot increases the GPU memory reservation.
```bash
# Example: explicitly use the recommended 2 hardware decoders
vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \
--media-io-kwargs '{"video": {"backend": "pynvvideocodec", "hw_decoders": 2}}'
```
#### Video Frame Recovery
For improved robustness when processing potentially corrupted or truncated video files, vLLM supports optional frame recovery using a dynamic window forward-scan approach. When enabled, if a target frame fails to load during sequential reading, the next successfully grabbed frame (before the next target frame) will be used in its place.