iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
5,799 Commits 3 Branches 0 Tags
57504a4bcf82cf544a4261778fc0505e492e4484
Commit Graph
50 Commits
Author SHA1 Message Date
Cyrus Leung 56d4aefa33 [VLM] Avoid unnecessary dummy multimodal data during processing (#16416) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 19:32:14 +00:00
Cyrus Leung 83b824c8b4 [VLM] Remove BaseProcessingInfo.get_mm_max_tokens_per_item (#16408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 09:06:58 -07:00
Roger WangDarkLight1337mgoinJennifer Zhao
 		f2ebb6f541 [V1] Scatter and gather placeholders in the model runner (#16076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
 2025-04-08 10:43:41 +08:00
Isotr0py fc0f87768a [Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-07 04:07:15 +00:00
Roger Wang af51d80fa1 Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus LeungmgoinRoger Wang
 		f5722a5052 [V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Bella kira f4c98b4d4c [Misc] Consolidate LRUCache implementations (#15481) 
Signed-off-by: Bella kira <2374035698@qq.com>
 2025-03-27 06:43:43 +00:00
Cyrus Leung ffa443afed [Bugfix] Fix embedding assignment for InternVL-based models (#15086) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-20 03:40:13 +00:00
Cyrus Leung 3d446433ec [Bugfix] Fix size calculation of processing cache (#15114) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 05:53:19 -07:00
Cyrus Leung 61f412187d [Bugfix] Re-enable Gemma3 for V1 (#14980) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-18 23:58:22 -07:00
Rémi DelacourtandDarkLight1337 61c6a5a796 [VLM] Merged multi-modal processor for Pixtral (#12211) 
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 06:28:27 -07:00
Cyrus Leung 3556a41434 [VLM] Limit multimodal input cache by memory (#14805) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 02:52:05 -07:00
Cyrus Leung ab93f1360f [VLM] Various cleanup and fixes (#14806) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-14 05:58:19 -07:00
Cyrus Leung 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-07 10:33:38 +00:00
Roger Wang ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-05 07:37:16 +00:00
Cyrus Leung f7bee5c815 [VLM][Bugfix] Enable specifying prompt target via index (#14038) 2025-02-28 07:35:55 -08:00
Cyrus Leung f1579b229d [VLM] Generalized prompt updates for multi-modal processor (#13964) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-27 17:44:25 +00:00
Isotr0py edf309ebbe [VLM] Support multimodal inputs for Florence-2 models (#13320) 2025-02-27 02:06:41 -08:00
Isotr0py ba5106e519 [LMM] Implement merged multimodal processor for whisper (#13278) 2025-02-23 01:46:03 -08:00
Cyrus Leung 4da1f667e9 [VLM] Keep track of whether prompt replacements have been applied (#13215) 2025-02-14 04:20:46 -08:00
Isotr0py bc55d13070 [VLM] Implement merged multimodal processor for Mllama (#11427) 2025-02-12 20:26:21 -08:00
Cyrus LeungandIsotr0py d1ca7df84d [VLM] Merged multi-modal processor for InternVL-based models (#12553) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-02-04 16:44:52 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus Leung 6609cdf019 [Doc] Add docs for prompt replacement (#12318) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-22 14:56:29 +00:00
Cyrus Leung cbdc4ad5a5 [Ci/Build] Fix mypy errors on main (#12296) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-22 12:06:54 +08:00
Cyrus Leung df76e5af26 [VLM] Simplify post-processing of replacement info (#12269) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 16:48:13 -08:00
Cyrus Leung 96912550c8 [Misc] Rename MultiModalInputsV2 -> MultiModalInputs (#12244) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-21 07:31:19 +00:00
Cyrus Leung bb354e6b2d [Bugfix] Fix various bugs in multi-modal processor (#12031) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-14 12:16:11 +00:00
Cyrus Leung b844b99ad3 [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-10 03:24:00 +00:00
Cyrus Leung 5984499e47 [Doc] Expand Multimodal API Reference (#11852) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 17:14:14 +00:00
Cyrus Leung 2a0596bc48 [VLM] Reorganize profiling/processing-related code (#11812) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 18:59:58 +08:00
Roger WangandDarkLight1337 91b361ae89 [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 19:58:16 +00:00
Cyrus Leung 996357e480 [VLM] Separate out profiling-related logic (#11746) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 16:02:21 +08:00
Cyrus Leung eed11ebee9 [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-04 11:40:53 +00:00
Aurick QiaoandAurick Qiao e1a5c2f0a1 [Model] Whisper model implementation (#11280) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-01-03 16:39:19 +08:00
Cyrus Leung 23c1b10a4c [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-02 17:00:00 +08:00
Cyrus LeungandIsotr0py a115ac46b5 [VLM] Move supported limits and max tokens to merged multi-modal processor (#11669) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-01 15:44:42 +00:00
Roger WangDarkLight1337Isotr0py
 		e7c7c5e822 [V1][VLM] V1 support for selected single-image models. (#11632) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-12-31 21:17:22 +00:00
Cyrus Leung 8d9b6721e7 [VLM] Abstract out multi-modal data parsing in merged processor (#11620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-30 15:01:35 +00:00
Cyrus Leung 101418096f [VLM] Support caching in merged multi-modal processor (#11396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-27 17:22:48 +00:00
Isotr0pyCyrus LeungDarkLight1337
 		e24113a8fe [Model] Refactor Qwen2-VL to use merged multimodal processor (#11258) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 16:28:00 +00:00
Cyrus Leung 6142ef0ada [VLM] Merged multimodal processor for Qwen2-Audio (#11303) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 06:14:17 +00:00
Isotr0pyandCyrus Leung d927dbcd88 [Model] Refactor Ultravox to use merged input processor (#11198) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-16 10:09:53 +00:00
Cyrus Leung b10609e6a1 [Misc] Clean up multi-modal processor (#11207) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-15 06:30:28 +00:00
Cyrus Leung 93abf23a64 [VLM] Fully dynamic prompt replacement in merged input processor (#11199) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 17:52:18 +00:00
Isotr0pyandCyrus Leung a811dd6608 [Model] merged input processor for Phi-3-Vision models (#10977) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-09 12:55:10 -08:00
Cyrus Leung 39e227c7ae [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 17:10:05 +00:00
Cyrus LeungandRoger Wang 955fa9533a [3/N] Support and implement merged input processor for LLaVA model (#10676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-12-07 00:50:58 -08:00
Cyrus Leung c8acd80548 [2/N] handling placeholders in merged multi-modal processor (#10485) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-22 21:25:09 -08:00
Cyrus Leung 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00