iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
68 Commits
Author SHA1 Message Date
Cyrus Leung 392645454b [Refactor] Decouple TimingContext from InputProcessingContext (#35083) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-23 14:15:50 +00:00
Cyrus Leung 987506bca6 [Refactor] Simplify dummy data generation (#35025) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-22 20:55:27 -08:00
Cyrus Leung a0d8d944e2 [Renderer] Move MM Hash parsing into Renderer (#34711) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-17 19:18:55 -08:00
Cyrus Leung 73391a1baa [Renderer] Move InputPreprocessor into Renderer (1/2) (#34510) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-14 10:14:21 -08:00
Isotr0py 0ab06100f4 [Multimodal] Expose mm_processor_kwargs for DummyInputsBuilder (#34330) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-11 09:37:40 -08:00
Cyrus Leung 88c3e114d8 [Refactor] Move MM data parsing outside processor (#33408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 16:46:14 +00:00
Matthew Bonanni a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Isotr0py 9ad7f89f55 [Models]: Make Multimodal config implicit in ViT implementation (#31972) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-24 20:34:26 +08:00
Alex Brooks 27b81e010d [Bugfix] Fix Granite Vision / Don't use Siglip Pooling Head Nested Models by Default (#32299) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2026-01-21 11:11:52 +08:00
Cyrus Leung 193069d129 [5/N] Initialize MM components in context managers (Q-Z) (#32695) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-20 19:10:23 +00:00
Cyrus Leung cdba4c74b3 [Model] Avoid token selection in SigLIP pooling head (#32389) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-15 17:01:59 +08:00
Cyrus Leung 9ea07b41da [1/N] Reorganize multimodal processing code (#32327) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-14 15:25:31 +00:00
Cyrus Leung 583a90e005 [Refactor] Separate sequence and token pooling types (#32026) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 04:53:24 +00:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Cyrus Leung c8ed39b9dd [Model] Reorganize pooling layers (#31973) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-09 11:02:14 +00:00
Cyrus Leung 5576227bc1 [Model] Standardize common vision encoders (#31947) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 02:33:16 -08:00
Isotr0py 700a5ad6c6 [MM Encoder]: Migrate legacy ViT MultiHeadAttention to new MMEncoderAttention interface (#30684) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 02:04:19 +08:00
Cyrus Leung c46b932df2 [Chore] Deprecate SupportsMultiModal.merge_by_field_config (#30170) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 07:57:28 +00:00
Cyrus Leung 64bc09ba27 [Core] Enable inputs_embeds_size separate from hidden_size (#29741) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-30 17:31:12 +08:00
Isotr0py e4bb2684bc [Models] Replace all nn.Conv2d with vLLM's Conv2dLayer (#28842) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-18 18:56:04 +00:00
Luciano MartinsLuciano MartinsIsotr0py
 		c2612371ad [Model] Add Gemma3 GGUF multimodal support (#27772) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-18 08:56:29 -08:00
Harry Mellor 97d1c99302 Rename clashing method names for vLLM model protocol (#27583) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 19:14:33 -08:00
Yu Jiaqi 15be507c86 [bugfix] fix siglip batch text output error (#28365) 
Signed-off-by: piood <2477084691@qq.com>
 2025-11-10 21:21:15 +08:00
Yu Jiaqi 4f882be4a0 [Model] Siglip2 Model Support (#27566) 
Signed-off-by: piood <2477084691@qq.com>
 2025-10-27 06:57:37 -07:00
Yu Jiaqi 0552cfb195 [Model] Siglip Embedding Support (#27324) 
Signed-off-by: piood <2477084691@qq.com>
 2025-10-23 20:19:48 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung d7e34b4210 [Model] Move vision_feature_select_strategy into resolve_visual_encoder_outputs (#25938) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-30 11:24:57 +00:00
ZhiyuandMichael Goin 431535b522 Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 22:40:33 +00:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Lukas Geiger c3bb9f2331 [Model] Use in-place adds in SigLIP (#18922) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-05-30 17:12:59 +08:00
Harry Mellor 26d0419309 Update deprecated type hinting in models (#18132) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 22:06:50 -07:00
Cyrus Leung 83b824c8b4 [VLM] Remove BaseProcessingInfo.get_mm_max_tokens_per_item (#16408) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-10 09:06:58 -07:00
Jerry Zhang dcf2a590f5 Allow torchao quantization in SiglipMLP (#15575) 2025-03-26 22:45:51 -07:00
Isotr0py 1477ffc381 [VLM] Cleanup siglip legacy code and fix broken paligemma multimodal processor (#14602) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 11:27:36 +00:00
Alex Brooks 983a40a8bb [Bugfix] Fix Positive Feature Layers in Llava Models (#13514) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
 2025-02-19 08:50:07 +00:00
Isotr0py fa253f1a70 [VLM] Remove input processor from clip and siglip (#13165) 2025-02-13 00:31:37 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Kevin H. Luu 64ea24d0b3 [ci/lint] Add back default arg for pre-commit (#12279) 
Signed-off-by: kevin <kevin@anyscale.com>
 2025-01-22 01:15:27 +00:00
Jee Jee Li 1f1542afa9 [Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-01-21 07:49:08 +00:00
Cyrus Leung ca47e176af [Misc] Move some model utils into vision file (#11848) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 17:04:46 +00:00
Cyrus Leung eed11ebee9 [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-04 11:40:53 +00:00
Cyrus Leung 8c38ee7007 [VLM] Merged multi-modal processor for LLaVA-NeXT (#11682) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-02 16:39:27 +00:00
Isotr0py 10398b4706 [Model] Consolidate ViTs attention implementation without mask (#10893) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-12-04 18:11:08 +00:00
Alex BrooksandCyrus Leung 1cfde82ffd [Model] Add Support for Multimodal Granite Models (#10291) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-21 10:46:20 +00:00
Isotr0py e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
Isotr0py c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
litianjianlitianjianDarkLight1337
 		5f8d8075f9 [Model][VLM] Add multi-video support for LLaVA-Onevision (#8905) 
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-28 18:04:10 +00:00
Cyrus LeungandIsotr0py c18e1a3418 [VLM] Enable overriding whether post layernorm is used in vision encoder + fix quant args (#9217) 
Co-authored-by: Isotr0py <2037008807@qq.com>
 2024-10-23 11:27:37 +00:00