iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
50 Commits
Author SHA1 Message Date
Zhonghua Deng fadb8d5c2d [Bugfix]Change the exception thrown by call_hf_processor from RuntimeError to ValueError (#18181) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-05-15 09:01:47 -07:00
Harry Mellor d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Cyrus Leung ebb3930d28 [Misc] Move config fields to MultiModalConfig (#17343) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-29 06:37:21 +00:00
Cyrus Leung aec9674dbe [Core] Remove legacy input mapper/processor from V0 (#15686) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 15:38:48 +08:00
Cyrus Leung 355f66348c [V1] Remove legacy input registry (#15673) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 23:34:34 -07:00
Cyrus Leung 247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 17:36:32 +00:00
Cyrus Leung f6137adbcb Revert "[Bugfix] Limit profiling run sequence length by max_model_len (#14785) (#14892) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-16 09:13:46 -07:00
Kyle Sayers d30aa7e9e6 [Bugfix] Limit profiling run sequence length by max_model_len (#14785) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-03-16 07:44:19 -07:00
Cyrus LeungWoosuk KwonRoger Wang
 		382403921f [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-13 02:23:12 -07:00
Cyrus Leung 9513290032 [Misc] Upgrade to Python 3.9 typing for additional directories (#14492) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-08 17:35:50 +00:00
Isotr0py 609ef61fea [Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-08 16:52:34 +00:00
Cyrus Leung 82551ad616 [Core] Don't use cache during multi-modal profiling (#14336) 2025-03-06 08:03:31 -08:00
Cyrus Leung 377d10bd14 [VLM][Bugfix] Pass processor kwargs properly on init (#13516) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-19 13:13:50 +00:00
Isotr0py bc55d13070 [VLM] Implement merged multimodal processor for Mllama (#11427) 2025-02-12 20:26:21 -08:00
Isotr0pyandCyrus Leung 815079de8e [VLM] merged multimodal processor and V1 support for idefics3 (#12660) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-02-04 20:00:51 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus Leung 12664ddda5 [Doc] [1/N] Initial guide for merged multi-modal processor (#11925) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 14:30:25 +00:00
Cyrus LeungandMaxime Fournioux d848800e88 [Misc] Move print_*_once from utils to logger (#11298) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
 2025-01-09 12:48:12 +08:00
Cyrus Leung 2a0596bc48 [VLM] Reorganize profiling/processing-related code (#11812) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 18:59:58 +08:00
Tobias Pitters b6087a6bee [mypy] Pass type checking in vllm/inputs (#11680) 
Signed-off-by: Tobias Pitters <tobias.pitters@gmail.com>
 2025-01-02 16:18:15 +00:00
Cyrus LeungandIsotr0py a115ac46b5 [VLM] Move supported limits and max tokens to merged multi-modal processor (#11669) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-01 15:44:42 +00:00
Cyrus Leung 101418096f [VLM] Support caching in merged multi-modal processor (#11396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-27 17:22:48 +00:00
Rafael Vasquez 32aa2059ad [Docs] Convert rST to MyST (Markdown) (#11145) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-23 22:35:38 +00:00
Cyrus Leung 6142ef0ada [VLM] Merged multimodal processor for Qwen2-Audio (#11303) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 06:14:17 +00:00
Cyrus Leung 93abf23a64 [VLM] Fully dynamic prompt replacement in merged input processor (#11199) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 17:52:18 +00:00
Isotr0pyandCyrus Leung a811dd6608 [Model] merged input processor for Phi-3-Vision models (#10977) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-09 12:55:10 -08:00
Cyrus LeungandRoger Wang 955fa9533a [3/N] Support and implement merged input processor for LLaVA model (#10676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-07 00:50:58 -08:00
Cyrus Leung 133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Cyrus Leung 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
Cyrus Leung bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-05 10:07:31 +08:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
Cyrus Leung cee711fdbb [Core] Rename input data types (#8688) 2024-10-16 10:49:37 +00:00
Alex Brooks a3691b6b5e [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-08 14:12:56 +00:00
Cyrus LeungandRoger Wang e1a3f5e831 [CI/Build] Update models tests & examples (#8874) 
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-09-28 09:54:35 -07:00
Michael Goin b28d2104de [Misc] Change dummy profiling and BOS fallback warns to log once (#8820) 2024-09-26 16:18:14 -07:00
Chen Zhangsimon-moChang SuSimon MoRoger WangRoger Wang
 		770ec6024f [Model] Add support for the multi-modal Llama 3.2 model (#8811) 
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-09-25 13:29:32 -07:00
Alex Brooks 9b8c8ba119 [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-09-23 07:44:48 +00:00
Cyrus Leung 5e85f4f82a [VLM] Use SequenceData.from_token_counts to create dummy data (#8687) 2024-09-20 23:28:56 -07:00
Cyrus Leung 0455c46ed4 [Core] Factor out common code in SequenceData and Sequence (#8675) 2024-09-21 02:30:39 +00:00
zifeitong df1a21131d [Model] Fix Phi-3.5-vision-instruct 'num_crops' issue (#7710) 2024-08-22 09:36:24 +08:00
SangBin Cho ff7ec82c4d [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) 2024-08-18 17:57:20 -07:00
Roger Wang bbf55c4805 [VLM] Refactor MultiModalConfig initialization and profiling (#7530) 2024-08-17 13:30:55 -07:00
Cyrus Leung 3f674a49b5 [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) 2024-08-14 17:55:42 +00:00
Cyrus Leung 9ba85bc152 [mypy] Misc. typing improvements (#7417) 2024-08-13 09:20:20 +08:00
Cyrus Leung 9389380015 [Doc] Move guide for multimodal model and other improvements (#6168) 2024-07-06 17:18:59 +08:00
Cyrus Leung ae96ef8fbd [VLM] Calculate maximum number of multi-modal tokens by model (#6121) 2024-07-04 16:37:23 -07:00
xwjiang2010andRoger Wang d9e98f42e4 [vlm] Remove vision language config. (#6089) 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-07-03 22:14:16 +00:00
Cyrus LeungXiaowei Jiangywang96xwjiang2010Roger Wang
 		9831aec49f [Core] Dynamic image size support for VLMs (#5276) 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-07-02 20:34:00 -07:00
xwjiang2010Cyrus LeungRoger Wang
 		98d6682cd1 [VLM] Remove image_input_type from VLM config (#5852) 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-07-02 07:57:09 +00:00
Cyrus Leungandywang96 5cbe8d155c [Core] Registry for processing model inputs (#5214) 
Co-authored-by: ywang96 <ywang@roblox.com>
 2024-06-28 12:09:56 +00:00