Commit Graph
217 Commits
Author SHA1 Message Date
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
waizuichougouandNick Hill 3f1af35e01 [Bugfix][Multimodal] Handle prefix-covered items in SHM worker cache (#54994)
Signed-off-by: waizuichougou <2082431897@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-09-03 06:39:34 +00:00
waizuichougou c6bca6e585 [Bugfix][Multimodal] Scope cache hash kwargs by modality (#54918)
Signed-off-by: waizuichougou <2082431897@qq.com>
2026-09-02 14:20:26 +00:00
Jan Kwieciński 1f1f628859 [Feat][MM Hashing] include media_io_kwargs in multi-modal hashes (#54241)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
2026-09-01 13:17:01 +00:00
Jan Kwieciński ec32f669bb [Feature][MM_UUIDs] Allow empty video URLs when using multi-modal UUIDs (#54220)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
2026-09-01 05:55:27 +00:00
Prudhvi Vuda 7292ee2791 [Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs (#53808)
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
2026-08-31 06:37:12 +00:00
Isotr0py 4fc943b867 [Multimodal] Deprecate PyAV video decoder backend (#54231)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-08-29 15:13:28 +00:00
Moe Huzaifa 299ebd094a [Bugfix][Multimodal] Reject malformed base64 audio with 400 instead of 500 (#53744)
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
2026-08-25 17:31:42 +00:00
Cyrus Leung ecfa7bb373 [MM] Cache common token sequences (#53560)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-08-24 07:48:10 -07:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
2026-08-24 01:15:09 +00:00
Cyrus LeungandQwen Code ebcd606467 [MM] Simplify prompt updates: replace PromptSeq with list[int] (#53372)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Qwen Code <noreply@alibaba-inc.com>
2026-08-23 14:50:32 +08:00
Yiqinandshenyiqin cb09dd7488 [Core][Multimodal] Skip redundant placeholder scan when token match succeeds (#52925)
Co-authored-by: shenyiqin <shenyiqin1@huawei.com>
2026-08-20 15:04:29 +00:00
Prudhvi Vuda de216b6e64 [Bugfix] Skip MM processor cache inserts larger than capacity (#53016)
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
2026-08-20 13:42:50 +00:00
9b2aef5124 [Bugfix] Video loading: sample over presentable frames, not header sample count (MP4 edit-list trims) (#48608)
Signed-off-by: AmitMY <amitmoryossef@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-08-20 04:23:09 +00:00
Isotr0py 3bb9c18f0c [Multimodal] Reorganize video decoder backends (#49155)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-08-18 15:06:18 +00:00
Juan Pérez de Algaba ceb340e2eb fix: prevent PyNvVideoCodec decoder slot limit bypass via ClassVar shadowing (#52126)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-08-17 17:45:40 +00:00
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-08-14 09:27:24 +00:00
Kevin H. LuuandOpenAI Codex 8eb35c5217 [CI] Mirror external test assets in vLLM S3 (#52064)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-08-13 02:08:59 +00:00
dmai-afk 4c51ceb11b [Bugfix][Multimodal] Invalidate retained PyNvVideoCodec decoder after failure (#51139)
Signed-off-by: Dat Mai <dmai@nvidia.com>
2026-08-12 11:02:10 +00:00
dmai-afk 86e2ab50aa [Bugfix][Frontend] Return 400 for invalid PyNvVideoCodec video input (#51120)
Signed-off-by: Dat Mai <dmai@nvidia.com>
2026-08-12 10:58:46 +00:00
3962042304 [Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747)
Signed-off-by: William Zhao <williamzh@meta.com>
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-12 12:06:57 +08:00
Tianyu GuoandOpenAI Codex 3fb7bb46d6 [Perf] Avoid repeated multimodal prompt update scans (#51774)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-08-11 09:23:24 +00:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-10 15:49:17 +00:00
Junpu Yu d3621c1eb3 [Bugfix][Multimodal] Fix PyNvVideoCodec video backend returning NCHW instead of NHWC (#51076)
Signed-off-by: Junpu Yu <davidyu@nvidia.com>
2026-08-08 04:49:06 +00:00
Vineeth Sai Varikuntla 7f2e78ba4d [Bugfix] Emit a valid media type from encode_{audio,image,video}_url (#49056)
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
2026-08-03 19:02:23 +00:00
Hareesh GaliandClaude Fable 5 89ac407e3d [Perf] Speed up multimodal placeholder and token-match scanning (#50716)
Signed-off-by: Hareesh <hgsata@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 08:27:56 +00:00
Juan Pérez de Algaba d83eb0b36b fix(security): classify DeepStream as GPU backend and enforce pixel limits (#50755)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-08-03 08:23:09 +00:00
Nicolò Lucchesi 17beffd55f [Misc] Clarify mono audio requirement (#50141)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
2026-07-31 10:46:53 +00:00
f727951d3f [Bugfix] Re-land MiniMax M3 default video processor (#50305)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-07-31 07:46:18 +00:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-31 05:25:19 +00:00
Nick Hill 2ecd8645d8 Revert "[Misc][Minimax-M3]add default video_processor (#50092)" (#50313) 2026-07-29 13:46:07 -07:00
Guan-Ming Chiu aeaa50a71c [Bugfix][Multimodal] Include media IO config in MM cache hash (#49975)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
2026-07-29 10:48:40 +00:00
rongfu.lengandCyrus Leung df2735ea2e [Misc][Minimax-M3]add default video_processor (#50092)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-07-29 01:24:51 -07:00
Zach Zhu 58f9659397 [Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665)
Signed-off-by: Zach Zhu <zzqshu@126.com>
2026-07-29 04:20:40 +00:00
nvbfalk 60915c972c [Feature] Add VidCom2 video token pruning (#47750)
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
2026-07-28 03:20:17 +00:00
Guan-Ming ChiuIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
04502deca2 [Perf] Hash videos by source bytes (#49607)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Signed-off-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 23:57:55 +08:00
Brandon PelfreyandOpenAI Codex 94682b79f4 [multimodal] Make PyNvVideoCodec decoder concurrency configurable (#49753)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-07-24 23:31:41 -07:00
Thomas FahrnerandClaude Fable 5 d02df748bf [Bugfix] Accept RFC 2397 parameters in base64 data URLs (#48973)
Signed-off-by: Thomas Fahrner <thomas.fahrner@parasail.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 08:23:08 -07:00
Isotr0py 1750e443f2 [Misc] Move PyNvVideoCodec stuff out of gpu worker (#49322)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-22 10:11:56 +08:00
bastefaniak 4d30c510ce [bugfix] Fix Cosmos3 Edge checkpoint weights filtering, video loading, prompt expansion (#49190)
Signed-off-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
2026-07-21 17:18:36 +08:00
Isotr0py 15cb8e140d [Multimodal] Allow keeping original image mode for ImageIO (#49159)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-07-20 13:42:45 +00:00
700e882eab Add TorchCodec as a video decoding backend (#46609)
Signed-off-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2026-07-06 19:58:51 -07:00
Alexis K. 40cc2e8327 [Bugfix] Return HTTP 422 for unprocessable image URLs instead of 500 (#47165)
Signed-off-by: Alexis Kinsella <alexis.kinsella@gmail.com>
2026-07-06 11:56:23 +00:00
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ba22152096 fix(security): block request-level GPU video backend selection withou… (#47259)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 02:36:49 -07:00
VectorPeak 68294739d1 [Bugfix] Align OpenCV video metadata timeline (#47099)
Signed-off-by: VectorPeak <73048950+VectorPeak@users.noreply.github.com>
2026-06-30 20:43:42 +00:00
Juan Pérez de Algaba 364ee36af1 fix(security): prevent image decompression bomb OOM denial of service (#47010)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-06-30 09:39:22 +00:00
Brandon PelfreyandRoger Wang af16446bf3 Vram semaphore infra (#44465)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
2026-06-26 17:32:51 -07:00
peizhang56andClaude 35a49fcfc2 [CI][Bugfix] Spawn engine in mm cache sleep test to fix ROCm HIP error (#46749)
Signed-off-by: pei.zhang <pei.zhang@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-26 00:38:26 -05:00
Matti4 1502cf6274 Fix relative allowed local media paths (#45263) 2026-06-25 20:45:20 -07:00
Juan Pérez de Algaba 83fa302ca4 fix(security): prevent infinite loop in split_audio with NaN audio sa… (#46463) 2026-06-23 10:24:51 +00:00