iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
8,621 Commits 3 Branches 0 Tags
070da660c1bf9e7a7be8b9efeff4b06f91c7342f
Commit Graph
211 Commits
Author SHA1 Message Date
Staszek Paśko 22341b996e Improve multimodal hasher performance for re-used Image prompts (#22825) 
Signed-off-by: Staszek Pasko <staszek@gmail.com>
 2025-08-15 12:32:56 +00:00
Sayandip DuttaandCyrus Leung aa300c438d [Bugfix] Unquote file uri before reading image (#22912) 
Signed-off-by: Sayandip Dutta <sayandip199309@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-15 09:28:00 +00:00
Cyrus Leung 19b927e52d [Core] Use individual MM items in P0/P1 cache and model runner (#22570) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-13 07:18:07 -07:00
milesialandAlexandre Milesi 20d65aa755 [Frontend] Multithreaded async multimodal load_bytes (#22710) 
Signed-off-by: Alexandre Milesi <30204471+milesial@users.noreply.github.com>
Co-authored-by: Alexandre Milesi <30204471+milesial@users.noreply.github.com>
 2025-08-13 06:09:26 -07:00
Cyrus Leung 951b038298 [Misc] Move jsontree to utils (#22622) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-11 03:49:32 -07:00
Isotr0py fbd8595c5c [Bugfix] Fix basic models tests hanging due to mm processor creation (#22571) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-09 11:42:21 -07:00
+10
Roger WangNing XieTJianAndrew SansomZhiyuShu WangXIn LiJunhao LiChaunceyYuxuan ZhangZiTian ZhaoCyrus LeungPo-Han HuangiAmir97iAmir97gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus LeungHong Hanhyoukaichaolkchen
 		08b751ba74 Implicit language-model-only mode via limit-mm-per-prompt (#22299) 
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>
 2025-08-08 22:21:40 -07:00
Cyrus Leung 1712543df6 [CI/Build] Fix multimodal tests (#22491) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-08 00:31:19 -07:00
Cyrus Leung 139d155781 [Frontend] Use engine argument to control MM cache size (#22441) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 09:47:10 -07:00
Cyrus Leung 8c9da6be22 [Core] Simplify mm processing cache (#22457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 09:47:07 -07:00
Cyrus Leung 766bc8162c [Core] Store only the keys for multi-modal data in P0 (#22198) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 01:45:04 -07:00
Cyrus Leung f5d0f4784f [Frontend] Improve error message for too many mm items (#22114) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-02 02:20:38 -07:00
GamhangandJinheng Li 0a6d305e0f feat(multimodal): Add customizable background color for RGBA to RGB conversion (#22052) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
Co-authored-by: Jinheng Li <ahengljh@gmail.com>
 2025-08-01 06:07:33 -07:00
Cyrus Leung 9532a6d563 [Deprecation] Remove deprecated args and methods (#21907) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-30 23:46:38 -07:00
milesialandAlexandre Milesi 0e36abf993 [Bugfix] Correct max tokens for non-contiguous embeds (#21798) 
Signed-off-by: Alexandre Milesi <30204471+milesial@users.noreply.github.com>
Co-authored-by: Alexandre Milesi <30204471+milesial@users.noreply.github.com>
 2025-07-30 01:16:25 +00:00
Christian Pinto 8560a5b258 [Core][Model] PrithviMAE Enablement on vLLM v1 engine (#20577) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2025-07-23 11:00:23 -07:00
Sergio Paniego Blanco 6364af92f8 Fixed typo in profiling logs (#21441) 2025-07-23 01:18:54 -07:00
Cyrus Leung 32dffc2772 [Core] Rename get_max_tokens_per_item for backward compatibility (#20630) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-08 23:11:30 +00:00
Jee Jee Li 906e05d840 [Misc] Remove the unused LoRA test code (#20494) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-05 13:48:16 +08:00
Michael Goin c108781c85 [CI Bugfix] Fix pre-commit failures on main (#20502) 2025-07-04 14:17:30 -07:00
Chenheli Hua ffe00ef77a [Misc] Small: Remove global media connector. Each test should have its own test connector object. (#20395) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-04 08:15:03 -07:00
Chenheli Hua b616f6a53d [Misc] Small: Fix video loader return type annotations. (#20389) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-03 03:10:39 +00:00
Chenheli Hua 2e7cbf2d7d [Frontend] Support configurable mm placeholder strings & flexible video sampling policies via CLI flags. (#20105) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-01 23:34:03 -07:00
Yuxuan ZhangandIsotr0py ed70f3c64f Add GLM4.1V model (Draft) (#19331) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-01 12:48:26 +00:00
Kyle Sayers d8cf819a9a [Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
h-avsha 3443aaf8dd Move to a faster base64 implementation (#19984) 
Signed-off-by: h-avsha <avshalom.manevich@hcompany.ai>
 2025-06-24 20:33:51 -07:00
Woosuk KwonandRoger Wang 2c5302fadd [Multimodal] Optimize Qwen2/2.5-VL startup time (#19756) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-06-21 20:01:07 +00:00
Lukas Geiger 3b523e38d9 [Core] Do not copy array during hashing (#19484) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-18 15:36:55 -07:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Lukas Geiger 96ada386b7 [Misc] Remove unused MultiModalHasher.hash_prompt_mm_data (#19422) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-11 05:18:57 +00:00
Lukas Geiger 319cb1e351 [Core] Batch multi modal input using pinned memory (#19169) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-10 13:44:59 +08:00
Guillaume Calmettes 9bc8bb07cf [Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided (#19202) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-06-05 12:59:28 +00:00
Lukas Geiger 1409ef9134 [Core] Cast multimodal input in hf processor (#18862) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-03 20:24:56 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
jennyyyyzhenyZhenyzhen
 		ebb1ec9318 [Model] enable data parallel for Llama4 vision encoder (#18368) 
Signed-off-by: yzhen <yzhen@devgpu093.cco2.facebook.com>
Co-authored-by: yZhen <yZhen@fb.com>
Co-authored-by: yzhen <yzhen@devgpu093.cco2.facebook.com>
 2025-06-02 19:22:54 +08:00
Cyrus Leung 7f2c1a87e9 [Deprecation] Require overriding get_dummy_text and get_dummy_mm_data (#18796) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-28 15:08:35 +08:00
Cyrus Leung 696259ca01 [Core] Automatically cast multi-modal input dtype (#18756) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-27 23:45:48 +08:00
Hyogeun Oh (오효근) a68e293cb9 [Doc] Convert Sphinx directives ( {class}, {meth}, {attr}, ...) to MkDocs format for better documentation linking (#18663) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-05-27 01:44:20 -07:00
Lukas Geiger 6071e989df [Core][Multimodal] Convert PIL Image to array without data copy when hashing (#18682) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-05-25 17:33:35 +00:00
Cyrus Leung 57fd13a707 [Bugfix] Fix profiling dummy data for Pixtral (#18677) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-25 14:05:30 +00:00
Feng XiaoLongandCrucifixion-Fxl 4fc1bf813a [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
Cyrus Leung 273cb3b4d9 [Doc] Fix top-level API links/docs (#18621) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-23 09:46:56 -07:00
Harry Mellor 2edb533af2 Replace {func} with mkdocs style links (#18610) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 05:51:38 -07:00
Harry Mellor a1fe24d961 Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Chenheli Hua 04eb88dc80 Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-05-23 01:59:18 +00:00
Lukas Geiger 54631f8262 [Misc] Call ndarray.tobytes() directly instead of ndarray.data.tobytes() (#18347) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-05-22 09:00:13 -07:00
Chenheli Hua 4f07a64075 Support custom implementations of VideoLoader backends. (#18091) 2025-05-15 13:26:49 +08:00
David XiaandAaron Pham 749f792553 [Frontend] decrease import time of vllm.multimodal (#18031) 
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
 2025-05-14 15:43:32 -07:00
Cyrus Leung 61e0a506a3 [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-12 22:40:19 -07:00
Isotr0py be8ff88e66 [Bugfix] Fix Video IO error for short video (#17791) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-07 15:36:06 +00:00