iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
5,799 Commits 3 Branches 0 Tags
57504a4bcf82cf544a4261778fc0505e492e4484
Commit Graph
148 Commits
Author SHA1 Message Date
Cyrus Leung 56d4aefa33 [VLM] Avoid unnecessary dummy multimodal data during processing (#16416) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 19:32:14 +00:00
Cyrus Leung 83b824c8b4 [VLM] Remove BaseProcessingInfo.get_mm_max_tokens_per_item (#16408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 09:06:58 -07:00
Jintao 4aed0ca6a2 [bugfix] Avoid the time consumption caused by creating dummy videos. (#16371) 2025-04-10 04:30:05 +00:00
Cyrus Leung 4ebc0b9640 [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-08 09:45:21 -07:00
Roger WangDarkLight1337mgoinJennifer Zhao
 		f2ebb6f541 [V1] Scatter and gather placeholders in the model runner (#16076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-04-08 10:43:41 +08:00
Gregory Shtrasberg ad434d4cfe Print the warning only once (#16193) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-04-07 18:30:06 +00:00
Isotr0py fc0f87768a [Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-07 04:07:15 +00:00
Roger Wang af51d80fa1 Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus LeungmgoinRoger Wang
 		f5722a5052 [V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Isotr0pyandHarry Mellor 4e5a0f6ae2 [Misc] Allow using OpenCV as video IO fallback (#15055) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-01 15:55:13 +00:00
Isotr0py 0a298ea418 [Bugfix] Fix no video/image profiling edge case for MultiModalDataParser (#15828) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-01 18:17:11 +08:00
Cyrus Leung 803d5c35f3 [V1] Override mm_counts for dummy data creation (#15703) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-30 03:20:42 -07:00
Roger Wang c67abd614f [V1] Support interleaved modality items (#15605) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-29 06:30:09 -07:00
Cyrus Leung c6bc0034d0 [Misc] Remove unused utils and clean up imports (#15708) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-28 09:41:16 -07:00
Cyrus Leung 355f66348c [V1] Remove legacy input registry (#15673) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 23:34:34 -07:00
Cyrus Leung 8693e47e6a [Bugfix] Fix mm_hashes forgetting to be passed (#15668) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-28 05:51:05 +00:00
Bella kira f4c98b4d4c [Misc] Consolidate LRUCache implementations (#15481) 
Signed-off-by: Bella kira <2374035698@qq.com>
 2025-03-27 06:43:43 +00:00
cyyever 1aa162e030 Apply torchfix (#15532) 
Signed-off-by: cyy <cyyever@outlook.com>
 2025-03-26 12:09:06 +00:00
Cyrus Leung a9e879b316 [Misc] Clean up MiniCPM-V/O code (#15337) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 10:22:52 +00:00
Cyrus Leung ffa443afed [Bugfix] Fix embedding assignment for InternVL-based models (#15086) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-20 03:40:13 +00:00
Cyrus Leung 3d446433ec [Bugfix] Fix size calculation of processing cache (#15114) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 05:53:19 -07:00
Cyrus Leung 1fe0fd12d3 [Misc] Avoid unnecessary HF do_rescale warning when passing dummy data (#15107) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 03:42:31 -07:00
Cyrus Leung 61f412187d [Bugfix] Re-enable Gemma3 for V1 (#14980) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-18 23:58:22 -07:00
Cyrus Leung 166a168b0f [Doc] Fix misleading log during multi-modal profiling (#14955) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-17 06:14:32 -07:00
Rémi DelacourtandDarkLight1337 61c6a5a796 [VLM] Merged multi-modal processor for Pixtral (#12211) 
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 06:28:27 -07:00
Cyrus Leung 3556a41434 [VLM] Limit multimodal input cache by memory (#14805) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 02:52:05 -07:00
Cyrus Leung ab93f1360f [VLM] Various cleanup and fixes (#14806) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-14 05:58:19 -07:00
Cyrus LeungWoosuk KwonRoger Wang
 		382403921f [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-13 02:23:12 -07:00
Chauncey b0746fae3d [Frontend] support image embeds (#13955) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-03-10 12:36:03 +00:00
Cyrus Leung 9513290032 [Misc] Upgrade to Python 3.9 typing for additional directories (#14492) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-08 17:35:50 +00:00
Isotr0py 609ef61fea [Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-08 16:52:34 +00:00
Cyrus Leung 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-07 10:33:38 +00:00
Cyrus Leung 82551ad616 [Core] Don't use cache during multi-modal profiling (#14336) 2025-03-06 08:03:31 -08:00
Roger Wang ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-05 07:37:16 +00:00
Cyrus Leung f7bee5c815 [VLM][Bugfix] Enable specifying prompt target via index (#14038) 2025-02-28 07:35:55 -08:00
Cyrus Leung f1579b229d [VLM] Generalized prompt updates for multi-modal processor (#13964) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-27 17:44:25 +00:00
Isotr0py edf309ebbe [VLM] Support multimodal inputs for Florence-2 models (#13320) 2025-02-27 02:06:41 -08:00
cjackal 51010a1807 [Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Isotr0py ba5106e519 [LMM] Implement merged multimodal processor for whisper (#13278) 2025-02-23 01:46:03 -08:00
Cyrus Leung 377d10bd14 [VLM][Bugfix] Pass processor kwargs properly on init (#13516) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-19 13:13:50 +00:00
Cyrus Leung 7b623fca0b [VLM] Check required fields before initializing field config in DictEmbeddingItems (#13380) 2025-02-17 01:36:07 -08:00
Cyrus Leung 4da1f667e9 [VLM] Keep track of whether prompt replacements have been applied (#13215) 2025-02-14 04:20:46 -08:00
Roger Wang dd5ede4440 [V1] Consolidate MM cache size to vllm.envs (#13239) 2025-02-13 20:19:03 -08:00
Cyrus Leung c9d3ecf016 [VLM] Merged multi-modal processor for Molmo (#12966) 2025-02-13 04:34:00 -08:00
Isotr0py bc55d13070 [VLM] Implement merged multimodal processor for Mllama (#11427) 2025-02-12 20:26:21 -08:00
Cyrus Leung 75404d041b [VLM] Update compatibility with transformers 4.49 2025-02-05 19:09:45 -08:00
Cyrus Leung a4ce74c14a [VLM] Use shared field to pass token ids to model 2025-02-05 13:30:46 -08:00
Kero Liang 62467a834a Avoid unnecessary multi-modal input data copy when len(batch) == 1 (#12722) 
Signed-off-by: imkero <kerorek@outlook.com>
 2025-02-04 21:03:19 +08:00
Cyrus LeungandIsotr0py d1ca7df84d [VLM] Merged multi-modal processor for InternVL-based models (#12553) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-02-04 16:44:52 +08:00
youkaichao 20579c0fae make sure mistral_common not imported for non-mistral models (#12669) 
When people use deepseek models, they find that they need to solve cv2
version conflict, see https://zhuanlan.zhihu.com/p/21064432691 .

I added the check, and make all imports of `cv2` lazy.

---------

Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-03 13:40:25 +08:00