Commit Graph
551 Commits
Author SHA1 Message Date
Juan Pérez de AlgabaandOpenAI Codex 7173c64e2e [Security] Bound decoded and downloaded media per request (#61226)
Signed-off-by: juanpas97 <juan.perezalgaba@gmail.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-13 10:17:14 +00:00
092c35dd15 [Model] Support Cloudflare/clef-omni on /v1/systemone (#61064)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-10-12 21:27:34 +08:00
Bruno VolpatoandCodex 61f084b5f3 [Bugfix][Multimodal] Fix per-video cache option reuse (#55203)
Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 06:21:55 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
Vadim GimpelsonandClaude ee406adf0c [Bugfix] Fix ZeroDivisionError in Qwen3-VL video sampling when the source fps is unknown (#60501)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-08 06:28:44 +00:00
Luciano MartinsandLuciano Martins 02b83919aa [Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
2026-10-06 13:43:38 -04:00
Juan Pérez de Algaba 710ac56e69 [Security] Restrict Pillow image formats on untrusted media paths (#60022)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
2026-10-05 10:40:15 +00:00
0872ddf4b9 [Bugfix][Multimodal] Normalize single-channel audio to 1D (#56691)
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-04 06:11:41 +00:00
tenotekandClaude Fable 5.1 443dccd3c9 [Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695)
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 22:14:13 +08:00
Harry MellorandClaude Opus 5 e3cae8d2ac [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor (#57387)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-02 02:25:18 +00:00
3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-30 05:16:26 -07:00
Isotr0py 31876d8ea1 [Bugfix] Fix TorchCodec audio IO correctness (#58364)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 11:34:48 -05:00
Daniel M. García-Ocaña Hernández 295a0b8d64 [Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution (#56372)
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com>
2026-09-29 03:58:39 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 14:14:10 +00:00
Tianyu GuoandCyrus Leung 3ee2906f47 [EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-23 09:03:50 +08:00
Cyrus Leung 7853700d4c [MM] Move get_dummy_processor_inputs into MM processor (#57967)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-21 09:39:20 -07:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-21 07:43:07 -07:00
Juan Pérez de AlgabaandCursor 7268f6e38e [Security] Prefer fresh multimodal payloads over a stale receiver cache (#57833)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-21 12:50:48 +00:00
Cyrus Leung 751f6807d9 [Refactor] Use dynamic MM cache in processor (#57674)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-19 15:38:00 +00:00
Hotragn PettuganiClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a8d1aa9c99 [Bugfix][Multimodal] Frame the multi-modal hash digest input (#54283)
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-18 21:06:48 -07:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 17:29:11 +01:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Cyrus Leung 0b9e018fb9 [MM] Further cleanup _apply_hf_processor_main (#53610)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2026-09-16 10:21:00 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Ama Senevirathneandamasen02 9ca6dbba71 fix(multimodal): tolerate malformed EXIF metadata during hashing (#56527) (#56576)
Signed-off-by: amasen02 <amasen02@users.noreply.github.com>
Co-authored-by: amasen02 <amasen02@users.noreply.github.com>
2026-09-16 07:56:50 +00:00
Juan Pérez de AlgabaandCursor ea723c81c3 [Security] Cap Qwen-VL video sampling knobs (#56729)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-14 10:22:18 +00:00
Hotragn PettuganiandIsotr0py a2685f2cda [Bugfix][Multimodal] Validate base64 video payloads, matching image and audio (#54323)
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
2026-09-14 04:14:26 +00:00
Isotr0py 22258a26bc [Multimodal] Use GPU NVDEC for EPD encoder-only instance video media IO (#53675)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-11 22:20:11 +08:00
waizuichougouIsotr0pyIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
295ac4e52e [Bugfix][Multimodal] Parse decoded video frame lists as a single video (#55326)
Signed-off-by: waizuichougou <2082431897@qq.com>
Signed-off-by: waizuichougou <105572299+waizuichougou@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-11 12:16:05 +00:00
Hung Hoang (黄兴)andCursor Agent 5ff50f3996 [Bugfix][MM] Fix swapped H/W in dummy video profiling inputs (#56385)
Signed-off-by: hungh <hungh@nvidia.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-09-11 05:25:51 +00:00
Jiatai WangandOpenAI Codex a36dfc93cb [Bugfix][Multimodal] Restore cached audio inputs with UUIDs (#56310)
Signed-off-by: JiataiWang <wangjiatai@proton.me>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-10 19:20:03 +00:00
Jared WenIsotr0pymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
912d2e79fe [multimodal][feat] add torchaudio backend to AudioResampler (#52598)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 06:42:38 +00:00
Harry MellorandClaude Opus 5 719284fe15 [Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count (#56078)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 09:23:30 -07:00
Adababyandshaolila 94848eda60 [Bugfix] Fix unreachable None guard in Molmo2 get_candidate_target_fps (#55893)
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
2026-09-09 12:37:10 +00:00
Andreas Karatzas f7f060d253 [Bugfix][Audio] Restore soundfile-first automatic decoding (#55642)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
2026-09-07 06:21:02 +00:00
lucamotzandOpenAI Codex f4eccdadef [Bugfix][Multimodal] Bound renderer warmup to the prefill token budget (#55448)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-05 16:43:28 +00:00
Sun c81ace1859 [Perf] Read VidCom2 frame budgets once (#55331)
Signed-off-by: levius <2114377220@qq.com>
2026-09-04 20:11:42 +00:00
6a039f465e [feat] add torchcodec as audio loader and implement selective audio backend (#51826)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: saltman155 <saltman155@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-04 16:16:10 +00:00
Juan Pérez de AlgabaandIsotr0py 8b6de0eb9a [Security] Cap GLMGA video sampling to prevent request-driven resource exhaustion (#54935)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-04 12:02:41 +00:00
Luyi Xiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8a728663c1 [Bugfix] Reject tokenizer-less Qwen VL processor init (#54886)
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-03 21:25:35 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
waizuichougouandNick Hill 3f1af35e01 [Bugfix][Multimodal] Handle prefix-covered items in SHM worker cache (#54994)
Signed-off-by: waizuichougou <2082431897@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-09-03 06:39:34 +00:00
waizuichougou c6bca6e585 [Bugfix][Multimodal] Scope cache hash kwargs by modality (#54918)
Signed-off-by: waizuichougou <2082431897@qq.com>
2026-09-02 14:20:26 +00:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a566ea7e8f [Perf] Avoid more h2d copies from non-pinned tensors (#54660)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 20:37:09 -07:00
Jan Kwieciński 1f1f628859 [Feat][MM Hashing] include media_io_kwargs in multi-modal hashes (#54241)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
2026-09-01 13:17:01 +00:00
lzhan011andCyrus Leung ce7391712b [Bugfix][Security] Bound embedding densification before to_dense() (#54632)
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-09-01 08:45:14 +00:00
Jan Kwieciński ec32f669bb [Feature][MM_UUIDs] Allow empty video URLs when using multi-modal UUIDs (#54220)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
2026-09-01 05:55:27 +00:00
Prudhvi Vuda 7292ee2791 [Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs (#53808)
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
2026-08-31 06:37:12 +00:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-08-31 03:20:47 +00:00
waizuichougouandOpenAI Codex f6895a5fcb [Bugfix][Multimodal] Avoid caching full prompts in fallback (#54439)
Signed-off-by: waizuichougou <2082431897@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-08-30 16:19:38 +00:00