55 Commits
Author SHA1 Message Date
alpandmottopanikeiku 2268d83bd9 [Bugfix][Multimodal] Honor RGB PNG transparency when loading images (#61986)
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
2026-10-16 23:09:12 +00:00
a9fe30e9f6 [Bugfix][Multimodal] Recover PyNvVideoCodec decoder slots after failures (#59597)
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Signed-off-by: danielafrimi <45691845+danielafrimi@users.noreply.github.com>
Signed-off-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Daniel Afrimi <dafrimi@oci-hsg-cs-001-login-02.cm.cluster>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
2026-10-15 09:17:19 +03:00
Juan Pérez de AlgabaandOpenAI Codex 7173c64e2e [Security] Bound decoded and downloaded media per request (#61226)
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-13 10:17:14 +00:00
Juan Pérez de Algaba 710ac56e69 [Security] Restrict Pillow image formats on untrusted media paths (#60022)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
2026-10-05 10:40:15 +00:00
Isotr0py 31876d8ea1 [Bugfix] Fix TorchCodec audio IO correctness (#58364)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 11:34:48 -05:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Hotragn PettuganiandIsotr0py a2685f2cda [Bugfix][Multimodal] Validate base64 video payloads, matching image and audio (#54323)
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
2026-09-14 04:14:26 +00:00
Isotr0py 22258a26bc [Multimodal] Use GPU NVDEC for EPD encoder-only instance video media IO (#53675)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-11 22:20:11 +08:00
Andreas Karatzas f7f060d253 [Bugfix][Audio] Restore soundfile-first automatic decoding (#55642)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-07 06:21:02 +00:00
6a039f465e [feat] add torchcodec as audio loader and implement selective audio backend (#51826)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: saltman155 <saltman155@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-04 16:16:10 +00:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632)
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-01 08:45:14 +00:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-08-31 03:20:47 +00:00
Moe Huzaifa 299ebd094a [Bugfix][Multimodal] Reject malformed base64 audio with 400 instead of 500 (#53744)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 17:31:42 +00:00
Juan Pérez de Algaba 0d7d5ed0b2 fix(security): enforce VLLM_MAX_AUDIO_CLIP_FILESIZE_MB on all audio paths (#53561)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-08-24 21:37:02 +00:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
2026-08-24 01:15:09 +00:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-10 15:49:17 +00:00
Juan Pérez de Algaba d83eb0b36b fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-08-03 08:23:09 +00:00
Guan-Ming Chiu aeaa50a71c [Bugfix][Multimodal] Include media IO config in MM cache hash (#49975)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
2026-07-29 10:48:40 +00:00
Guan-Ming ChiuIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
04502deca2 [Perf] Hash videos by source bytes (#49607)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Signed-off-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 23:57:55 +08:00
Brandon PelfreyandOpenAI Codex 94682b79f4 [multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-24 23:31:41 -07:00
Thomas FahrnerandClaude Fable 5 d02df748bf [Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
Signed-off-by: Thomas Fahrner <thomas.fahrner@parasail.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 08:23:08 -07:00
Guan-Ming Chiu 80c7683923 [Perf] Defer MM embeds loading off the event loop (#49477)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
2026-07-23 13:58:50 +00:00
Isotr0py 15cb8e140d [Multimodal] Allow keeping original image mode for ImageIO (#49159)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-20 13:42:45 +00:00
Juan Pérez de Algaba 793cf79c89 [Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation (#48583)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-07-14 11:08:24 +00:00
Alexis K. 40cc2e8327 [Bugfix] Return HTTP 422 for unprocessable image URLs instead of 500 (#47165)
Signed-off-by: Alexis Kinsella <alexis.kinsella@gmail.com>
2026-07-06 11:56:23 +00:00
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ba22152096 fix(security): block request-level GPU video backend selection withou… (#47259)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 02:36:49 -07:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-30 09:39:22 +00:00
Matti4 1502cf6274 Fix relative allowed local media paths (#45263) 2026-06-25 20:45:20 -07:00
Juan Pérez de Algaba 3d20275bb4 fix(security): enforce audio decode duration limit in chat completions path (#45908)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-17 11:07:13 +00:00
Juan Pérez de Algaba f715f25f29 Fix misleading error for audio duration limit rejection (#45113)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-12 09:58:08 +00:00
Juan Pérez de Algaba cf1c906724 [Security] Fix image EXIF orientation and tRNS transparency handling (#44974)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-09 09:34:44 -07:00
Juan Pérez de Algaba 1b1359c332 [Security] Fix DoS via audio decompression bomb in speech-to-text endpoint (#44970)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-10 00:18:53 +08:00
littlecircle0730andHsiao-Yuan Chen d8218b1ee7 [Bugfix] Propagate ImportError from load_audio_pyav when vllm[audio] … (#44750)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Co-authored-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
2026-06-09 04:24:52 +00:00
Isotr0py f8e9c56d15 [Multimodal] Automatically select registered video loader for VLM (#44126)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-06-02 09:09:47 +00:00
Lukas Geiger 597ed13803 [Core][MM] Do not use urllib3 to parse data URLs (#42535)
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
2026-05-13 22:21:01 +00:00
storyiconandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6aa057c9d7 [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133)
Signed-off-by: storyicon <storyicon@foxmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 15:50:04 +08:00
Nick CaoandClaude 153ba7f0f3 [Refactor] Drop direct dependency on librosa (#39079)
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-04-18 06:55:38 +00:00
Isotr0py 82531edbfb [Refactor] Remove resampy dependency (#39524)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-04-16 08:48:17 -07:00
Yiyang "Ian" LiuandCopilot 18b1c77211 fix: handle ImportError in load_audio (#39473)
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-14 19:09:06 +08:00
Juan Pérez de Algaba 58ee614221 (security) Enforce frame limit in VideoMediaIO (#38636)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-04-01 10:23:45 +00:00
Andreas Karatzas 677424c7ac [Core][CI] Add opt-in media URL caching via VLLM_MEDIA_CACHE (#37123)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-03-30 04:58:53 -07:00
aliialsaeedii 7e76af14fa [Bugfix][Frontend] Return 400 for corrupt/truncated image inputs instead of 500 (#38253)
Signed-off-by: aliialsaeedii <ali.al-saeedi@nscale.com>
2026-03-30 10:26:46 +00:00
70a2152830 [MultiModal] add support for numpy array embeddings (#38119)
Signed-off-by: guillaume_guy <guillaume.guy@airbnb.com>
Signed-off-by: Guillaume Guy <guillaume.c.guy@gmail.com>
Co-authored-by: guillaume_guy <guillaume.guy@airbnb.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2026-03-25 20:13:04 +00:00
Andreas Karatzas 5dac719b2b [Bugfix] Handle libsndfile sf_error(NULL) race condition in audio fallback (#37782)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-03-22 13:37:29 +08:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-03-21 11:36:15 +08:00
918b7890a1 [Bugfix] Fix base64 JPEG video frames returning empty metadata (#37301)
Signed-off-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-03-18 13:40:03 +00:00
Isotr0py a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-03-17 14:44:19 +00:00
Tianyu GuoandIsotr0py 5578f2a4d3 Support online use_audio_in_video (#36319)
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-03-09 07:16:44 -07:00
milesial 755356b3d1 feat: expose media_io_kwargs at runtime (#34778)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
2026-03-07 04:27:04 +00:00
+2 5d199ac8f2 Support Audio Extraction from MP4 Video for Nemotron Nano VL (#35539)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Andrii <askliar@nvidia.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Andrii Skliar <askliar@oci-nrt-cs-001-vscode-01.cm.cluster>
Co-authored-by: Andrii <askliar@nvidia.com>
Co-authored-by: root <root@pool0-03748.cm.cluster>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: root <root@pool0-02416.cm.cluster>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: root <root@pool0-04880.cm.cluster>
2026-03-03 23:20:33 -08:00