[multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com> Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
0ba2aa35a8
commit
94682b79f4
@@ -818,16 +818,18 @@ Full example: [examples/generate/multimodal/openai_chat_completion_client_for_mu
|
||||
|
||||
#### Video Decoding Backend
|
||||
|
||||
vLLM decodes video bytes into frames using a selectable decoding backend. Three
|
||||
vLLM decodes video bytes into frames using a selectable decoding backend. Five
|
||||
backends are supported:
|
||||
|
||||
- `opencv` (default): OpenCV-based decoder.
|
||||
- `pyav`: PyAV decoder.
|
||||
- `torchcodec`: TorchCodec (PyTorch-native) decoder.
|
||||
- `pynvvideocodec`: NVIDIA NVDEC-based decoder.
|
||||
- `deepstream`: NVIDIA DeepStream NVDEC-based decoder.
|
||||
|
||||
All three backends are ultimately backed by FFmpeg. `torchcodec` lets
|
||||
you choose which FFmpeg version is used while `opencv` and `pyav` rely on
|
||||
whichever FFmpeg build they were linked against.
|
||||
The CPU backends are backed by FFmpeg. `torchcodec` lets you choose which FFmpeg
|
||||
version is used while `opencv` and `pyav` rely on whichever FFmpeg build they
|
||||
were linked against.
|
||||
|
||||
Select the backend by passing the `backend` parameter via `--media-io-kwargs`:
|
||||
|
||||
@@ -854,6 +856,21 @@ vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \
|
||||
--media-io-kwargs '{"video": {"backend": "torchcodec", "seek_mode": "approximate", "num_ffmpeg_threads": 4}}'
|
||||
```
|
||||
|
||||
**PyNvVideoCodec-specific parameters:**
|
||||
|
||||
- `hw_decoders`: Maximum number of concurrent hardware decoder slots retained
|
||||
by each API server process. It must be a positive integer and defaults to `2`,
|
||||
which is the recommended starting point for concurrent video workloads.
|
||||
Because vLLM reserves GPU memory for these slots at startup, this value cannot
|
||||
be overridden per request. Benchmark before increasing it because each
|
||||
additional slot increases the GPU memory reservation.
|
||||
|
||||
```bash
|
||||
# Example: explicitly use the recommended 2 hardware decoders
|
||||
vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct \
|
||||
--media-io-kwargs '{"video": {"backend": "pynvvideocodec", "hw_decoders": 2}}'
|
||||
```
|
||||
|
||||
#### Video Frame Recovery
|
||||
|
||||
For improved robustness when processing potentially corrupted or truncated video files, vLLM supports optional frame recovery using a dynamic window forward-scan approach. When enabled, if a target frame fails to load during sequential reading, the next successfully grabbed frame (before the next target frame) will be used in its place.
|
||||
|
||||
Reference in New Issue
Block a user