iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
142 Commits
Author SHA1 Message Date
Aaron Pham 9ea7f1abf3 fix(regression): clone from reference items (#18662) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-05-24 15:25:20 +00:00
Harry Mellor 2edb533af2 Replace {func} with mkdocs style links (#18610) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 05:51:38 -07:00
Madeesh KannanandRussell Bryant e493e48524 [V0][Bugfix] Fix parallel sampling performance regression when guided decoding is enabled (#17731) 
Signed-off-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-05-23 03:38:23 -07:00
Nan QinandAndrew Sansom 9609327fa4 [Core] [Bugfix]: tensor parallel with prompt embeds (#18171) 
Signed-off-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
 2025-05-19 20:21:27 -07:00
wwl2755 9da1095daf [Spec Decode][V0] Fix spec decode correctness test in V0 eagle/medusa (#18175) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-05-18 19:49:46 -07:00
Harry Mellor d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Andrew Sansom临景Bryce1010Nan2018Cyrus LeungDarkLight1337
 		cc2a77d7f1 [Core] [Bugfix] Add Input Embeddings (#15428) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-02 01:06:39 -07:00
Cyrus Leung aec9674dbe [Core] Remove legacy input mapper/processor from V0 (#15686) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-28 15:38:48 +08:00
Bryan Lu 9ed6ee92d6 [Bugfix] EAGLE output norm bug (#14464) 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
 2025-03-15 06:50:33 +00:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Lucia FangandLiuXiaoxuanPKU f525c0be8b [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-19 17:06:23 +08:00
Cody Yu 6ac485a953 [V1][PP] Fix intermediate tensor values (#13417) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-02-17 13:37:45 -08:00
Xu Song 83481ceb49 [Bugfix] Fix missing parentheses (#13263) 2025-02-14 01:07:10 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Andy Lo 18fd4a8331 [Bugfix] Multi-sequence broken (#11898) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2025-01-21 11:51:35 -08:00
youkaichao f12141170a [torch.compile] consider relevant code in compilation cache (#11614) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-08 10:46:43 +00:00
Aurick QiaoandAurick Qiao e1a5c2f0a1 [Model] Whisper model implementation (#11280) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-01-03 16:39:19 +08:00
Michael Goin 74fa1d123c [Bugfix] Fix OpenAI parallel sampling when using xgrammar (#11637) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-31 03:43:54 +00:00
Rajveer BachkaniwalaandRoger Wang b5cbe8eeb3 [Bugfix] Last token measurement fix (#11376) 
Signed-off-by: rajveerb <46040700+rajveerb@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-12-28 11:34:46 +08:00
Sungjae Lee c31d4a57a6 [Core] support LoRA and prompt adapter in content-based hashing for Block Manager v2 prefix caching (#8240) 2024-12-13 07:51:25 -08:00
Cyrus Leung eeec9e3390 [Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
Cyrus Leung cad5c0a6ed [Doc] Update docs to refer to pooling models (#11093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-11 13:36:27 +00:00
Maximilien de BayserandFlavia Beo 214efc2c3c Support Cross encoder models (#10400) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-24 18:56:20 -08:00
Ricky Xu 4634a89d18 Prefix Cache Aware Scheduling [1/n] (#10128) 
Signed-off-by: rickyx <rickyx@anyscale.com>
 2024-11-22 21:15:55 -08:00
Cyrus Leung 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
zifeitong 47db6ec831 [Frontend] Add per-request number of cached token stats (#10174) 2024-11-12 16:42:28 +00:00
Cyrus Leung bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-05 10:07:31 +08:00
Gregory Shtrasberg b67feb1274 [Bugfix]Using the correct type hints (#9885) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2024-11-04 06:19:51 +00:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
youkaichao 97b61bfae6 [misc] avoid circular import (#9765) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-28 20:51:23 +00:00
youkaichaoandZhuohan Li 4fdc581f9e [core] simplify seq group code (#9569) 
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2024-10-24 00:16:44 -07:00
youkaichao 76a5e13270 [core] move parallel sampling out from vllm core (#9302) 2024-10-22 00:31:44 +00:00
Russell Bryant 776dbd74f1 [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-16 22:55:59 +00:00
Cyrus Leung cee711fdbb [Core] Rename input data types (#8688) 2024-10-16 10:49:37 +00:00
Chang SuandNick Hill ba30942240 [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) 
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
 2024-10-15 15:40:43 -07:00
youkaichaoandBrendan Wong cbc2ef5529 [misc] hide best_of from engine (#9261) 
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
 2024-10-10 21:30:44 -07:00
youkaichao e4d652ea3e [torch.compile] integration with compilation control (#9058) 2024-10-10 12:39:36 -07:00
Alex Brooks a3691b6b5e [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-08 14:12:56 +00:00
youkaichao 18b296fdb2 [core] remove beam search from the core (#9105) 2024-10-07 05:47:04 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath c2ec430ab5 [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-09-27 13:32:07 -07:00
Chen Zhangsimon-moChang SuSimon MoRoger WangRoger Wang
 		770ec6024f [Model] Add support for the multi-modal Llama 3.2 model (#8811) 
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-09-25 13:29:32 -07:00
Archit Patke 6da1ab6b41 [Core] Adding Priority Scheduling (#5958) 2024-09-24 19:50:50 -07:00
Travis Johnson 01b6f9e1f0 [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-09-24 17:29:56 -07:00
Alexander Matveev 1a2aef3e59 Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) 2024-09-23 15:38:04 -07:00
Cyrus Leung 5e85f4f82a [VLM] Use SequenceData.from_token_counts to create dummy data (#8687) 2024-09-20 23:28:56 -07:00
Cyrus Leung 0455c46ed4 [Core] Factor out common code in SequenceData and Sequence (#8675) 2024-09-21 02:30:39 +00:00
Nick Hill 18e9e1f7b3 [HotFix] Fix final output truncation with stop string + streaming (#8468) 2024-09-13 11:31:12 -07:00
Nick Hill 551ce01078 [Core] Add engine option to return only deltas or final output (#7381) 2024-09-12 12:02:00 -07:00
Yang FanRoger WangDarkLight1337
 		3b7fea770f [Model][VLM] Add Qwen2-VL model support (#7905) 
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-09-11 09:31:19 -07:00
Alexander Matveev 6d646d08a2 [Core] Optimize Async + Multi-step (#8050) 2024-09-03 18:50:29 +00:00