iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
12,946 Commits 3 Branches 0 Tags
eb28e8068dd870283f1a4fe8251ed9254d6f477e
Commit Graph
66 Commits
Author SHA1 Message Date
Xingyu Liu 80221e1884 [BugFix]Fix eagle draft_model_config and add tests (#31753) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
 2026-01-12 23:09:36 -08:00
Cyrus Leung 583a90e005 [Refactor] Separate sequence and token pooling types (#32026) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 04:53:24 +00:00
Cyrus Leung d1b6fe007f [Chore] Further cleanup pooler (#31951) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 02:16:21 -08:00
Xingyu Liu 0eee877f67 [Core] Parse vLLM engine required fields from hf_config to model_arch_config (#28454) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
 2026-01-02 15:13:15 -08:00
Nick Hill bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00
Cyrus Leung 7e24e5d4d6 [Deprecation] Remove deprecated task, seed and MM settings (#30397) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:59:39 -08:00
Cyrus Leung 5a87d8b9b1 [Deprecation] Remove deprecated plugin and compilation fields for v0.13 release (#30396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:59:35 -08:00
wang.yuqi 9e77ffca3f [Model][7/N] Improve all pooling task | Deprecation as_reward_model. Extract hidden states prefer using new multi-vector retrieval API (#26686) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-08 08:10:09 +00:00
Cyrus Leung e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung 27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 74c4d80c6c [Model][6/N] Improve all pooling task | Support chunked prefill with ALL pooling (#27145) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-04 13:44:15 +00:00
Arpit KhandelwalandLuka Govedič d7284a2604 [Core] Rename PassConfig flags as per RFC #27995 (#29646) 
Signed-off-by: arpitkh101 <arpit5khandelwal@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-12-03 03:38:55 +00:00
Harry Mellor 951445a52d Remove default values from InitVars so that they're not stored (#29859) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-02 12:16:37 +00:00
Boyuan Feng 3b221cb661 [BugFix] respect VLLM_LOGGING_LEVEL in logger (#29761) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-12-02 07:49:16 +00:00
wang.yuqiandCyrus Leung f4b76056ee Improve enable chunked_prefill & prefix_caching logic. (#26623) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-27 22:05:48 -08:00
Morrison TurnanskyadabeytaLuka GovedičHarry Mellor
 		0838b52e2e [Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-27 01:55:58 -08:00
Harry Mellor a8b70304d6 Update rope_scaling to rope_parameters in preparation for Transformers v5 (#28542) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-19 09:06:36 -08:00
Michael Goin c6873c4e6d [UX] Support nested dicts in hf_overrides (#25727) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-07 11:19:16 +08:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
wang.yuqi 7f570f1caa [V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-25 11:26:31 +00:00
ahao-anyscale c8bde93367 [BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-09-23 18:13:32 -06:00
Harry Mellor 058525b997 Move PoolerConfig from config/__init__.py to config/pooler.py (#25181) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 11:02:55 +00:00
Woosuk Kwon 759ef49b15 Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> c4afdb69cc Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
Harry Mellor f36355abfd Move LoadConfig from config/__init__.py to config/load.py (#24566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-10 06:14:18 -07:00
wang.yuqi 84cf78acee [Model] Pooling models default to using chunked prefill & prefix caching if supported. (#20930) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-11 09:41:37 -07:00
Harry Mellor c49848396d Refactor sliding window configuration to Transformers best practice (#21927) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-09 20:50:48 -07:00
Cyrus LeungandHarry Mellor 86ae693f20 [Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00
22quinn 8632e831ba [Core] Add update_config RPC method (#20095) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-07-14 00:49:18 +00:00
Nicolò LucchesiandDarkLight1337 020f58abcd [Core] Support multiple tasks per model (#20771) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-12 19:40:11 -07:00
wang.yuqi 6f1229f91d [Model][2/N] Automatic conversion of CrossEncoding model (#19978) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-07-03 13:59:23 +00:00
WangHuaqiang ccbfb1d1c9 [Bugfix] Fix the max_seq_len limit of 16384 for DeepSeek models (#20322) 
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
 2025-07-02 12:53:36 +00:00
Lionel Villard c05596f1a3 [Perf] Validate @config in pre-commit instead of dynamically (#20200) 
Signed-off-by: Lionel Villard <villard@us.ibm.com>
 2025-07-01 05:10:28 -04:00
Ye (Charlotte) Qi b692e9cd07 [Misc] Fix skipped max-model-len validation when deriving max model length from tokenizer config (#19660) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-06-16 06:30:29 +00:00
Cyrus Leung a2142f0196 Support non-string values in JSON keys from CLI (#19471) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 09:34:04 +00:00
Richard Zou da511d54d8 Fix CompilationConfig repr (#19091) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-06 16:23:35 +08:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Jerry Zhang 109e15a335 Add pt_load_map_location to allow loading to cuda (#16869) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-05-01 23:23:42 -07:00
Harry Mellor 13698db634 Improve configs - ModelConfig (#17130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-30 10:38:22 +08:00
Harry Mellor 2ef5d106bb Improve literal dataclass field conversion to argparse argument (#17391) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-29 16:25:08 +00:00
Harry Mellor d27ea94034 Improve configs - TokenizerPoolConfig + DeviceConfig (#16603) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-17 11:19:42 +00:00
Harry Mellor 47512b3200 Default to generation_config from model (#12622) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-08 14:46:15 +08:00
Harry Mellor 76c89fcadd Use smaller embedding model when not testing model specifically (#13891) 2025-02-28 00:50:43 -08:00
Kevin H. Luu 2c5e637b57 [ci] Use env var to control whether to use S3 bucket in CI (#13634) 2025-02-22 19:19:45 -08:00
Kevin H. Luu a64a84433d [2/n][ci] S3: Use full model path (#13564) 
Signed-off-by: <>
 2025-02-20 01:20:15 -08:00
Kevin H. LuuandEC2 Default User d5d214ac7f [1/n][CI] Load models in CI from S3 instead of HF (#13205) 
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
 2025-02-19 07:34:59 +00:00
Harry Mellor f2b20fe491 Consolidate Llama model usage in tests (#13094) 2025-02-13 22:18:03 -08:00
Nicolò Lucchesi d84cef76eb [Frontend] Add /v1/audio/transcriptions OpenAI API endpoint (#12909) 2025-02-13 07:23:45 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Yanyi Liu ff7424f491 [Frontend] Support override generation config in args (#12409) 
Signed-off-by: liuyanyi <wolfsonliu@163.com>
 2025-01-29 01:41:01 -08:00