Commit Graph
465 Commits
Author SHA1 Message Date
Guan-Ming Chiu aeaa50a71c [Bugfix][Multimodal] Include media IO config in MM cache hash (#49975)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
2026-07-29 10:48:40 +00:00
Li, Jiang 90245f4190 [Bugfix] Fix multi-modal support on CPU MRV2 (#50073)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-07-28 06:09:48 +00:00
nvbfalk 60915c972c [Feature] Add VidCom2 video token pruning (#47750)
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
2026-07-28 03:20:17 +00:00
Guan-Ming ChiuIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
04502deca2 [Perf] Hash videos by source bytes (#49607)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Signed-off-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 23:57:55 +08:00
Brandon PelfreyandOpenAI Codex 94682b79f4 [multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-24 23:31:41 -07:00
Thomas FahrnerandClaude Fable 5 d02df748bf [Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
Signed-off-by: Thomas Fahrner <thomas.fahrner@parasail.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 08:23:08 -07:00
Guan-Ming Chiu 80c7683923 [Perf] Defer MM embeds loading off the event loop (#49477)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
2026-07-23 13:58:50 +00:00
Isotr0py 1750e443f2 [Misc] Move PyNvVideoCodec stuff out of gpu worker (#49322)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-22 10:11:56 +08:00
bastefaniak 4d30c510ce [bugfix] Fix Cosmos3 Edge checkpoint weights filtering, video loading, prompt expansion (#49190)
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
2026-07-21 17:18:36 +08:00
Isotr0py ea0e9c8f2e [MRV2] Add encoder cache profiling implementation (#47985)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-20 20:18:14 -07:00
Isotr0py 15cb8e140d [Multimodal] Allow keeping original image mode for ImageIO (#49159)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-20 13:42:45 +00:00
wendadawenandTianyu Guo 69d4f5ef63 [Bugfix][Multimodal] Fix Qwen3-Omni use_audio_in_video with mixed image/video inputs (#46213)
Signed-off-by: wendadawen <wendadawen@qq.com>
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
2026-07-17 08:31:16 +00:00
Juan Pérez de Algaba 793cf79c89 [Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation (#48583)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-07-14 11:08:24 +00:00
e23b19309b Deepstream video backend (#42424)
Signed-off-by: Viranjan Pagar <vpagar@nvidia.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-07-10 02:23:30 -07:00
Isotr0py 5e975eae1a [Bugfix] Avoid blocking model launching when no system ffmpeg available for TorchCodec (#47888)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-08 10:52:25 +08:00
700e882eab Add TorchCodec as a video decoding backend (#46609)
Signed-off-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-07-06 19:58:51 -07:00
Alexis K. 40cc2e8327 [Bugfix] Return HTTP 422 for unprocessable image URLs instead of 500 (#47165)
Signed-off-by: Alexis Kinsella <alexis.kinsella@gmail.com>
2026-07-06 11:56:23 +00:00
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ba22152096 fix(security): block request-level GPU video backend selection withou… (#47259)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 02:36:49 -07:00
Ting SUNandCyrus Leung 379950191f [Bugfix][Multimodal] Normalize direct PIL image inputs (#47566)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-07-03 14:27:14 -07:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-30 09:39:22 +00:00
Brandon PelfreyandRoger Wang af16446bf3 Vram semaphore infra (#44465)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-06-26 17:32:51 -07:00
Matti4 1502cf6274 Fix relative allowed local media paths (#45263) 2026-06-25 20:45:20 -07:00
Tae Jeong 62890e204c Fix duplicated logging when loading a corrupt or partial video (#46467)
Signed-off-by: hhhhhhhhhhhhhhhhho <man2719@naver.com>
2026-06-24 06:14:13 -07:00
LynnandClaude Opus 4.8 52fbe12283 [Perf][Multimodal] Avoid building a full timestamps list in video frame sampling (#46543)
Signed-off-by: Lynn <lynnhe02@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:38:27 +00:00
Juan Pérez de Algaba 83fa302ca4 fix(security): prevent infinite loop in split_audio with NaN audio sa… (#46463) 2026-06-23 10:24:51 +00:00
Ranranmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Isotr0py
635c38338a [Multimodal] Add Qwen2-VL/Qwen2.5-VL processor-mapped video loader (#45555)
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-21 18:56:50 +00:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-06-20 20:02:24 -07:00
Juan Pérez de Algaba 3d20275bb4 fix(security): enforce audio decode duration limit in chat completions path (#45908)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-17 11:07:13 +00:00
Andreas Karatzas 7e179e4bc0 [ROCm][CI] Gate incompatible HF references on Transformers v5 (#41532)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-06-16 20:34:11 +08:00
Isotr0py e3cfea2e1b [Multimodal] Add Qwen3-VL video loader (#44412)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-16 03:45:34 +00:00
Juan Pérez de Algaba f715f25f29 Fix misleading error for audio duration limit rejection (#45113)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-12 09:58:08 +00:00
Akshat katiyarandClaude 47930b59ca [Bugfix] Handle HWC images in ImageProcessorItems.get_image_size (#45057)
Signed-off-by: YellowFoxH4XOR <yellowfoxh4xor@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-10 05:35:50 +00:00
Juan Pérez de Algaba cf1c906724 [Security] Fix image EXIF orientation and tRNS transparency handling (#44974)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-09 09:34:44 -07:00
Juan Pérez de Algaba 1b1359c332 [Security] Fix DoS via audio decompression bomb in speech-to-text endpoint (#44970)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-10 00:18:53 +08:00
littlecircle0730andHsiao-Yuan Chen d8218b1ee7 [Bugfix] Propagate ImportError from load_audio_pyav when vllm[audio] … (#44750)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Co-authored-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
2026-06-09 04:24:52 +00:00
Andreas Karatzas f0f6805d8a [CI] Stabilize the multi-audio OpenAI server path (#44051)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-06-07 15:54:32 +08:00
Jared Wen 6ac69203e8 [videoloader] implement glm46v video loader (#44417)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
2026-06-07 06:27:20 +00:00
tc-mb 3dbb4e0ace [Bugfix] MiniCPM-V-4.6 video inference crash: placeholder count mismatches visual embedding count (#44509)
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
2026-06-04 08:22:30 -07:00
Andy Lo 95b1615ec9 [Perf] Improve multimodal item handling from O(n) to O(log n) per step (#44212)
Signed-off-by: Andy Lo <andy@mistral.ai>
2026-06-03 11:00:26 +00:00
Isotr0py f8e9c56d15 [Multimodal] Automatically select registered video loader for VLM (#44126)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-02 09:09:47 +00:00
212deff2ec [feat] add GlmgaProcessor specific logits in glm4_1v.py (#43575)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-05-29 02:56:02 +00:00
Nick Hill b82e908b4c [Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-05-19 10:35:54 -04:00
Ranran f3d5360591 [Bugfix][Multimodal] PyAV video backend returns keyframes labeled as targets (#42586)
Signed-off-by: Ranran <hzz5361@psu.edu>
2026-05-14 08:56:59 -07:00
Lukas Geiger 597ed13803 [Core][MM] Do not use urllib3 to parse data URLs (#42535)
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
2026-05-13 22:21:01 +00:00
JooHo LeeandJooHo Lee 16863072ca [Bugfix] Fix scipy audio resampling ratio (#42233)
Signed-off-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
Co-authored-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
2026-05-13 18:52:41 +08:00
zhrrr 7a08b34fbf [Model Runner V2] support qwen35 / mamba hybrid model (#35520)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
2026-05-07 09:31:05 -07:00
yzong-rhandwang.yuqi 20dcd984f9 [Bugfix] Fix RuntimeError: Already borrowed by adding thread-safe Hugging Face fast-tokenizer wrappers (#41181)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-05-05 14:04:01 +00:00
Avshalom Manevich a80d6f150c better logging for large uncachable items (#41145)
Signed-off-by: h-avsha <avshalom.manevich@hcompany.ai>
2026-04-29 09:48:47 -07:00
Moritz Sanft 2c06cf3486 [Bugfix] use served_model_name for multimodal error message (#41003)
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2026-04-27 08:22:35 -07:00
storyiconandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6aa057c9d7 [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133)
Signed-off-by: storyicon <storyicon@foxmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 15:50:04 +08:00