iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
5,271 Commits 3 Branches 0 Tags
cfbca8a2f2b3dafe210d0bffa944f00a40112ac0
Commit Graph
510 Commits
Author SHA1 Message Date
maobaolong 26dd972adb [FEAT]Support reset prefix cache by specified device (#15003) 2025-03-19 10:54:41 -07:00
Simon Mo 3b457143d2 [Bugfix] Register serializers for V0 MQ Engine (#15009) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-03-18 09:14:47 -04:00
Sebastian Schoennenbeck dd732028f5 [Bugfix][Frontend] Fix validation of logprobs in ChatCompletionRequest (#14352) 
Signed-off-by: Sebastian Schönnenbeck <sebastian.schoennenbeck@comma-soft.com>
 2025-03-18 05:50:05 -07:00
Jun Duan 74bc397b0a [Core] Expose API endpoint /is_sleeping (#14312) 
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
 2025-03-15 06:28:14 -07:00
Robert Shawrshaw@neuralmagic.com <rshaw@neuralmagic.com>Nicolò LucchesiTyler Michael SmithMichael Goin
 		d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
daniel-salib 73deea2fdb [Frontend] track server_load (#13950) 2025-03-14 09:53:17 -07:00
Russell Bryant 0b0d6421b2 [Frontend] Fix log message to use http vs https (#14774) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-14 09:21:09 -07:00
Guillaume Calmettes fd8e055ffb [BugFix]: properly catch templating error when preprocess input (#13976) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-03-14 05:58:34 -07:00
Cyrus Leung ab93f1360f [VLM] Various cleanup and fixes (#14806) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-14 05:58:19 -07:00
Cyrus Leung f53a0586b9 [Bugfix] Fix prompt format of GLM4V (#14539) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-13 11:37:17 +00:00
Woosuk KwonRoger WangDarkLight1337
 		c0c25e25fa [Model] Add support for Gemma 3 (#14660) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-12 08:36:33 -07:00
Randy ChenandCody Yu 36e0c8f7da [Feature] Add vllm bench CLI (#13993) 
Signed-off-by: Randy Chen <acad.randyjhc@gmail.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-03-12 00:31:48 +00:00
Woosuk Kwon 0967110e42 [Minor] Update the tqdm bar for parallel sampling (#14571) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-10 14:23:48 -07:00
Harry Mellor 3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-10 16:36:21 +00:00
Chauncey b0746fae3d [Frontend] support image embeds (#13955) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-03-10 12:36:03 +00:00
Yuchen Yan 73ae0b44e9 [Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (#12428) 
Signed-off-by: Yuchen Yan <740987012@qq.com>
 2025-03-08 20:14:53 -08:00
Harry Mellor 47512b3200 Default to generation_config from model (#12622) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-08 14:46:15 +08:00
மனோஜ்குமார் பழனிச்சாமி cc10281498 [Misc] Set default value of seed to None (#14274) 
Signed-off-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
 2025-03-07 10:40:01 +00:00
Harry Mellor bf0560bda9 Reinstate best_of for V0 (#14356) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-06 08:34:22 -08:00
Nicolò Lucchesi fa82b93853 [Frontend][Docs] Transcription API streaming (#13301) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-03-06 10:39:35 +00:00
VincentBrayden ZhongHarry Mellor
 		a4f1ee35d6 Deprecate best_of Sampling Parameter in anticipation for vLLM V1 (#13997) 
Signed-off-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-05 20:22:43 +00:00
Benjamin Chislett 32985bed7c [Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-03-05 06:30:40 +00:00
Congcong Chen 0a995d5434 [Model] New model support for Phi-4-multimodal-instruct (#14119) 2025-03-04 20:57:01 -08:00
Harry Mellor e5b2f1601a [Frontend] Do prompt_logprobs clamping for chat as well as completions (#14225) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-04 20:13:06 +00:00
Harry Mellor 9badee53de Fix performance when --generation-config is not None (#14223) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-04 20:59:22 +01:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Ce Gao bf33700ecd [v0][structured output] Support reasoning output (#12955) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
 2025-03-02 14:49:42 -05:00
Jee Jee Li 5157338ed9 [Misc] Improve LoRA spelling (#13831) 2025-02-25 23:43:01 -08:00
Wen Sun 6522d55b6f Fix /v1/audio/transcriptions Bad Request Error (#13811) 2025-02-25 06:03:33 -08:00
cjackal 51010a1807 [Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Keyun Tong 8db1b9d0a1 Support SSL Key Rotation in HTTP Server (#13495) 2025-02-22 05:17:44 -08:00
Cyrus Leung 7f6bae561c [CI/Build] Fix pre-commit errors (#13696) 2025-02-22 00:31:26 -08:00
Robin c6ed93860f [Bugfix][API Server] Fix invalid usage of 'ge' and 'le' in port valid… (#13672) 2025-02-21 22:05:28 -08:00
Keyun Tong 0ffdf8ce0c [HTTP Server] Make model param optional in request (#13568) 2025-02-21 21:55:50 -08:00
Gabriel Marinho 1c3c975766 [FEATURE] Enables /score endpoint for embedding models (#12846) 2025-02-20 22:09:47 -08:00
Roger Wang a30c093502 [Bugfix] Add mm_processor_kwargs to chat-related protocols (#13644) 2025-02-20 22:04:33 -08:00
Yuan Tang 3738e6fa80 [API Server] Add port number range validation (#13506) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-02-20 15:05:13 +08:00
燃 041e294716 [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) 2025-02-19 23:04:30 -08:00
youkaichaoandcennn ba81163997 [core] add sleep and wake up endpoint and v1 support (#12987) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
 2025-02-20 12:41:17 +08:00
zifeitong d3231cb436 [Bugfix] Handle content type with optional parameters (#13383) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2025-02-18 11:29:13 +00:00
Yuan Tang a1074b3efe [Bugfix] Only print out chat template when supplied (#13444) 2025-02-17 21:43:31 -08:00
Nicolò Lucchesi 579d7a63b2 [Bugfix][Docs] Fix offline Whisper (#13274) 2025-02-14 21:32:37 -08:00
Pooya Davoodi 185cc19f92 [Frontend] Optionally remove memory buffer used for uploading to URLs in run_batch (#12927) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
 2025-02-14 08:22:42 +00:00
Nicolò Lucchesi d84cef76eb [Frontend] Add /v1/audio/transcriptions OpenAI API endpoint (#12909) 2025-02-13 07:23:45 -08:00
Vaibhav Jain 37dfa60037 [Bugfix] Missing Content Type returns 500 Internal Server Error (#13193) 2025-02-13 06:52:22 -08:00
Russell Bryant 578087e56c [Frontend] Pass pre-created socket to uvicorn (#13113) 2025-02-13 00:51:46 -08:00
Russell Bryant d46d490c27 [Frontend] Move CLI code into vllm.cmd package (#12971) 2025-02-12 23:12:21 -08:00
LikeSundayLikeRain 04f50ad9d1 [Bugfix] deepseek_r1_reasoning_parser put reason content in wrong field in certain edge case (#13097) 2025-02-12 23:11:26 -08:00
Rafael Vasquez 314cfade02 [Frontend] Generate valid tool call IDs when using tokenizer-mode=mistral (#12332) 2025-02-12 08:29:56 -08:00
Keyun Tong 3ee696a63d [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) 
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
 2025-02-12 12:25:58 +08:00