iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,776 Commits 3 Branches 0 Tags
94de871546e8da687c08ed8a7e0a26531500d4bd
Commit Graph
64 Commits
Author SHA1 Message Date
Cyrus Leung 88c3e114d8 [Refactor] Move MM data parsing outside processor (#33408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 16:46:14 +00:00
Cyrus Leung 51550179fc [Refactor] Define MM data parser in processing info instead of processor itself (#33260) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-29 13:55:17 +08:00
Cyrus Leung dcd80206b7 [Chore] Update type annotation of input_ids in model forward (#33063) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 06:02:10 -08:00
Cyrus Leung 2b8a38b6d6 [Model] Extend collect_children and no_init_weights contexts (#32757) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 08:20:27 +00:00
Cyrus Leung 9ea07b41da [1/N] Reorganize multimodal processing code (#32327) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-14 15:25:31 +00:00
Jee Jee Libk-201prashanth058Anexdeus
 		ce1eafd1a5 [Core] Initialize LoRA support for tower and connector in multi-modal models (#26674) 
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: Anexdeus <5142168@mail.ru>
 2025-12-26 04:48:20 -08:00
Cyrus Leung c46b932df2 [Chore] Deprecate SupportsMultiModal.merge_by_field_config (#30170) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 07:57:28 +00:00
Isotr0pyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 0ec8422171 [Bugfix] Fix incorrect channel order for idefics3 in edge case (#29881) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-02 16:03:52 +00:00
Harry Mellor 97d1c99302 Rename clashing method names for vLLM model protocol (#27583) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 19:14:33 -08:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor 4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung 59a85c366e [Model] Use merge_by_field_config for MM models (H-L) (#26230) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-05 11:54:17 +08:00
Wenlong WangandHarry Mellor 79aa244678 [Multi Modal] Configurable MM Profiling (#25631) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-03 03:59:10 -07:00
Isotr0py bd51f78e39 [V0 Deprecation][Models] Remove all V0 condition for mm embeddings merge (#25331) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-09-29 14:09:18 +08:00
Cyrus LeungNickLuccheRoger Wang
 		27d7638b94 [Bugfix] Merge MM embeddings by index instead of token IDs (#16229) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-27 08:15:12 +00:00
Isotr0py 231c2c63e4 [Bugfix] Fix idefics3 tie_word_embeddings (#25454) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-23 10:06:48 +00:00
Woosuk Kwon 1c3ffdbecc [V0 Deprecation] Remove V0 sampling metadata (#25345) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 10:37:11 -07:00
whx 4a9375fe9d [Model] Pass param prefix to LLMHead (#24862) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-17 16:01:27 +08:00
Cyrus Leung 27e8d1ea3e [Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-18 09:52:00 +00:00
Cyrus Leung 82de9b9d46 [Misc] Automatically resolve HF processor init kwargs (#22005) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-31 22:44:10 -07:00
Benji Beck 3ea57a56d9 Migrate Idefics3ImagePixelInputs and Idefics3ImageEmbeddingInputs to … (#21683) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
 2025-07-27 22:37:23 -07:00
Brayden Zhong 75a99b98bf [Chore] Remove outdated transformers check (#20989) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-07-15 19:42:40 -07:00
Cyrus Leung b024a42e93 [Core] Move multimodal placeholder from chat utils to model definition (#20355) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-03 08:18:30 +00:00
Kyle Sayers d8cf819a9a [Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
Russell Bryant 14fdd21d39 [Core] More fixes to MultiModalEmbeddings type handling (#19715) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-06-18 22:48:29 +00:00
Russell Bryant 90f9c2eb5c [V1] Change return type on get_multimodal_embeddings() (#19446) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-06-16 13:32:15 -04:00
Cyrus Leung 01dc9a76db [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-04 04:49:20 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Harry Mellor 26d0419309 Update deprecated type hinting in models (#18132) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 22:06:50 -07:00
Woosuk Kwon b411418ff0 [Chore] Remove Sampler from Model Code (#17084) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-24 02:49:33 -07:00
Cyrus Leung 56d4aefa33 [VLM] Avoid unnecessary dummy multimodal data during processing (#16416) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 19:32:14 +00:00
Cyrus Leung 83b824c8b4 [VLM] Remove BaseProcessingInfo.get_mm_max_tokens_per_item (#16408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 09:06:58 -07:00
Nicolò Lucchesi d55244df31 [Model] Add SupportsMultiModal.get_language_model interface (#16007) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-09 04:12:54 -07:00
Chauncey 102bf967f0 [Model] Add smolvlm support (#16017) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-08 19:12:17 -07:00
Roger WangDarkLight1337mgoinJennifer Zhao
 		f2ebb6f541 [V1] Scatter and gather placeholders in the model runner (#16076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-04-08 10:43:41 +08:00
Roger Wang af51d80fa1 Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus LeungmgoinRoger Wang
 		f5722a5052 [V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Cyrus Leung 09e974d483 [Bugfix] Check dimensions of multimodal embeddings in V1 (#15816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-31 09:01:35 -07:00
Cyrus Leung 541d1df486 [Bugfix] embed_is_patch for Idefics3 (#15696) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-28 08:27:52 -07:00
Cyrus Leung 247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 17:36:32 +00:00
Cyrus Leung 601bd3268e [Misc] Clean up type annotation for SupportsMultiModal (#14794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-14 00:59:56 -07:00
Tyler Michael Smith 4f5b059f14 Clean up unused padding_idx variables across many model definitions (#13240) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-04 21:27:00 +00:00
lkchen b3cf368d79 [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) 2025-03-04 15:43:59 +00:00
Cyrus Leung f1579b229d [VLM] Generalized prompt updates for multi-modal processor (#13964) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-27 17:44:25 +00:00
Harry Mellor cdc1fa12eb Remove unused kwargs from model definitions (#13555) 2025-02-24 17:13:52 -08:00
Jee Jee Li 105b8ce4c0 [Misc] Reduce LoRA-related static variable (#13166) 2025-02-22 00:21:30 -08:00
Cyrus Leung 377d10bd14 [VLM][Bugfix] Pass processor kwargs properly on init (#13516) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-19 13:13:50 +00:00
Isotr0pyandCyrus Leung 815079de8e [VLM] merged multimodal processor and V1 support for idefics3 (#12660) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-02-04 20:00:51 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00