iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
11,384 Commits 3 Branches 0 Tags
5bdd15527770ef39cc4c3cdca008fb4f9cf8a15f
Commit Graph
61 Commits
Author SHA1 Message Date
wang.yuqi 3729ed00ba [Model] Add num_cached_tokens for PoolingRequestOutput (#27378) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-23 14:03:42 +08:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		1f633b8632 [Frontend][3/N] Improve all pooling task | Support binary embedding response (#27066) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-22 18:38:57 +08:00
Cyrus Leung d31f7844f8 [Misc] Move utils to avoid conflicts with stdlib, and move tests (#27169) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-19 05:20:55 -07:00
Cyrus Leung d2740fafbf [Chore] Separate out vllm.utils.collections (#26990) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 08:35:35 +00:00
Cyrus Leung f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
Cyrus Leung 828523ad8e [Chore] Separate out vllm.utils.async_utils (#26913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 15:33:00 +00:00
wang.yuqiMaximilien de BayserCyrus Leung
 		d2a7938582 [Frontend][1/N] Improve all pooling task | Support FP16 Embedding Base64 (Still uses fp32 by default). (#26414) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-13 19:06:43 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Cyrus Leung 4bdf7ac593 [Bugfix] Fix SHM cache initialization (#26427) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 02:48:04 -07:00
Harry Mellor 4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Isotr0py a42d2df75f [Frontend] Cache chat template kwargs resolution (#26227) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-04 15:32:30 +00:00
Zhuohan Li 6c47f6bfa4 [Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Harry Mellor c1eda615ba Fix model name included in responses (#24663) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:47:51 -07:00
Flora Feng 77f62613f9 Consolidate rendering parameters into RenderConfig dataclass (#24543) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 08:44:47 +00:00
Flora Feng 15cb047e25 Extend renderer with embedding support and integrate completion endpoint (#24405) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 01:46:46 +08:00
Flora Feng 0661cb9df3 Add renderer-based prompt processing for embedding and classification endpoints (#24356) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-07 08:26:48 +00:00
Jakub Smid 28f350e147 Support add_generation_prompt in embeddings endpoint with chat request (#23931) 
Signed-off-by: biba10 <jaksmid@seznam.cz>
 2025-09-03 10:47:55 +00:00
Gabriel Marinhogemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Max de Bayser
 		5b8077b8ac Fix wrong truncate_prompt_tokens type hint (#22761) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Gabriel Marinho <104592062+gmarinho2@users.noreply.github.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-30 20:39:38 +00:00
Chen Zhang 3210264421 [Frontend] Add --log-error-stack to print stack trace for error response (#22960) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-27 04:58:59 +00:00
KdumpCyrus LeungMaximilien de BayserDarkLight1337
 		653124bd46 [Frontend] Add chunked processing to handle long inputs in embedding models (#22280) 
Signed-off-by: x22x22 <wadeking@qq.com>
Signed-off-by: Kdump <rootshellexp@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-13 04:14:24 -07:00
kourosh hakhamaneshi 9fe98d4250 [Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2025-07-25 06:49:11 -07:00
Michael Goin 82ec66f514 [V0 Deprecation] Remove Prompt Adapters (#20588) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 16:36:48 -07:00
Cyrus Leung 45badd05d0 [Core] Set pooling params based on task and model (#21128) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-18 05:41:17 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Frieda Huang 9cea90eab4 [Frontend] Add /classify endpoint (#17032) 
Signed-off-by: Frieda (Jingying) Huang <jingyingfhuang@gmail.com>
 2025-05-11 07:57:07 +00:00
Gabriel MarinhoandMax de Bayser 1c2bc7ead0 Truncation control for embedding models (#14776) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-04-30 09:24:57 +08:00
wang.yuqi fbf722c6e6 [Frontend] support matryoshka representation / support embedding API dimensions (#16331) 2025-04-11 23:23:10 -07:00
Jason (Siyu) Zhu cec8c7d7f8 Refactor error handling for multiple exceptions in preprocessing (#15650) 
Signed-off-by: JasonZhu1313 <jasonchu13@outlook.com>
 2025-03-28 03:27:20 +00:00
Guillaume Calmettes fd8e055ffb [BugFix]: properly catch templating error when preprocess input (#13976) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-03-14 05:58:34 -07:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Keyun Tong 0ffdf8ce0c [HTTP Server] Make model param optional in request (#13568) 2025-02-21 21:55:50 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Joe Runde 4db72e57f6 [Bugfix][Refactor] Unify model management in frontend (#11660) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-01-01 02:21:51 +00:00
Cyrus Leung 9edca6bf8f [Frontend] Online Pooling API (#11457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 17:54:30 +08:00
Joe Runde 2d1b9baa8f [Bugfix] Fix request cancellation without polling (#11190) 2024-12-17 12:26:32 -08:00
Cyrus Leung eeec9e3390 [Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
Joe Runde 980ad394a8 [Frontend] Use request id from header (#10968) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-12-10 13:46:29 +08:00
Cyrus Leung d2f058e76c [Misc] Rename embedding classes to pooling (#10801) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 14:36:51 +08:00
tomeras91 395b1c7454 [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) 
Signed-off-by: Tomer Asida <tomera@ai21.com>
 2024-11-27 13:21:10 -08:00
Noam Gat 11fcf0e066 Remove token-adding chat embedding params (#10551) 
Signed-off-by: Noam Gat <noamgat@gmail.com>
 2024-11-21 23:59:47 -08:00
Cyrus Leung 32e46e000f [Frontend] Automatic detection of chat content format from AST (#9919) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-16 13:35:40 +08:00
Nick Hill 29862b884b [Frontend] Adjust try/except blocks in API impl (#10056) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2024-11-06 20:07:51 -08:00
Cyrus Leung 06386a64dd [Frontend] Chat-based Embeddings API (#9759) 2024-11-01 08:13:35 +00:00
Cyrus Leung 051eaf6db3 [Model] Add user-configurable task for models that support both generation and embedding (#9424) 2024-10-18 11:31:58 -07:00
Flávia Béo 0dcc8cbe5a Adds truncate_prompt_tokens param for embeddings creation (#8999) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
 2024-10-04 18:31:40 +00:00
Sebastian Schoennenbeck 35bd215168 [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) 2024-10-01 09:58:06 +00:00
Jiaxin Shan 260d40b5ea [Core] Support Lora lineage and base model metadata management (#6315) 2024-09-20 06:20:56 +00:00
Alexander MatveevNick Hillrshaw@neuralmagic.com <rshaw@neuralmagic.com>Robert ShawSimon Mo
 		7c7714d856 [Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157) 
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2024-09-18 13:56:58 +00:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 0b769992ec [Bugfix]: Use float32 for base64 embedding (#7855) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2024-08-26 03:16:38 +00:00