iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
8,621 Commits 3 Branches 0 Tags
070da660c1bf9e7a7be8b9efeff4b06f91c7342f
Commit Graph
576 Commits
Author SHA1 Message Date
CsrayzandHarry Mellor a0632a3e03 [Frontend] Expose do_log_stats interval to env (#22905) 
Signed-off-by: Csrayz <jover@cmbchina.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-15 13:00:20 +00:00
Roger Wang da2705198f [Misc] clear and separate error messages for input too long and input + max-tokens too long (#22803) 
Signed-off-by: Roger Wang <hey@rogerw.me>
 2025-08-13 07:22:56 -07:00
KdumpCyrus LeungMaximilien de BayserDarkLight1337
 		653124bd46 [Frontend] Add chunked processing to handle long inputs in embedding models (#22280) 
Signed-off-by: x22x22 <wadeking@qq.com>
Signed-off-by: Kdump <rootshellexp@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-13 04:14:24 -07:00
Chen Zhang ad344ef552 [gpt-oss] Small bug fixes for frontend (#22512) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-11 22:04:38 -07:00
Chen Zhang 95a935fc48 [gpt-oss] Support streaming in response API (#22431) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-11 17:46:59 -07:00
Harry Mellor bc1d02ac85 [Docs] Add comprehensive CLI reference for all large vllm subcommands (#22601) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-11 00:13:33 -07:00
Maximilien de Bayser 39052dbca8 Support token_type_ids in V1 with less code changes (#21985) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-10 22:54:59 -07:00
yyweiss baece8c3d2 [Frontend] Add unix domain socket support (#18097) 
Signed-off-by: <yyweiss@gmail.com>
Signed-off-by: yyw <yyweiss@gmail.com>
 2025-08-08 16:23:44 -07:00
Chen Zhang fe6d8257a1 [gpt-oss] Support tool call and implement MCP tool server (#22427) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-08 15:06:37 -07:00
Andrew Sansom e2c8f1edec [PERF] Use pybase64 to more quickly decode prompt embeddings (#22469) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-08-07 19:15:32 -07:00
Woosuk Kwon 399d2a10e2 Fix pre-commit error in main (#22462) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-07 08:54:39 -07:00
Chen Zhang 4815b00f54 [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-07 08:33:25 -07:00
Chen Zhang 4da8bf20d0 [Tool] Fix auto tool call (#22434) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-07 07:03:38 -07:00
Adrián García García 8e8e0b6af1 feat: Add --enable-log-outputs flag for logging model generations (#20707) 
Signed-off-by: Adrian Garcia <adrian.garcia@inceptionai.ai>
 2025-08-06 23:10:13 -07:00
Moritz Sanft 370661856b [Frontend] Update OpenAI error response to upstream format (#22099) 
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
 2025-08-06 23:06:00 -07:00
Chen ZhangandWoosuk Kwon f6278b6243 [gpt-oss] Convert user input to harmony format (#22402) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-06 20:56:02 -07:00
Lionel Villard ad6c655dde preload heavy modules when mp method is forkserver (#22214) 
Signed-off-by: Lionel Villard <villard@us.ibm.com>
 2025-08-06 20:33:24 -07:00
qscqesze 5e9455ae8f [Bugfix]: Fix the streaming output for function calls in the minimax (#22015) 
Signed-off-by: QscQ <qscqesze@gmail.com>
Signed-off-by: qingjun <qingjun@minimaxi.com>
 2025-08-06 20:30:27 -07:00
Chen Zhang 19c9365aa4 [gpt-oss] add demo tool server (#22393) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-06 17:47:14 -07:00
Woosuk KwonLiuXiaoxuanPKUsimon-moChen ZhangHongxia YangMinseok LeeYongye Zhu
 		ec7cb19224 [gpt-oss] Add loop for built-in tool call (#22374) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-08-06 10:32:21 -07:00
Woosuk Kwon f263a4b53f [gpt-oss] Support chat completion api (#22342) 2025-08-06 01:57:39 -07:00
wang.yuqi 586f286789 [Model] Pooling model activation supports per request control by PoolingParams (#20538) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-05 00:37:00 -07:00
tlipoca9 8a6e108e76 fix: kimi_k2 return empty tool call list (#22149) 
Signed-off-by: tlipoca9 <tlipoca9@gmail.com>
 2025-08-04 19:15:31 -07:00
Woosuk Kwon 9af654cc38 [Responses API] Ignore store=True and process the request by default (#22185) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-04 05:12:48 -07:00
Woosuk Kwon 6d98843b31 [Responses API] Disable response store by default (#22137) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-03 04:04:21 -07:00
Nick Hill 8d524ce79f [BugFix] Improve internal DP load balancing (#21617) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 19:45:27 -07:00
Harry Mellor 2d7b09b998 Deprecate --disable-log-requests and replace with --enable-log-requests (#21739) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-01 17:16:37 +01:00
Nick Hill 3146519add [BugFix] Don't change title of top-level process (#22032) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 07:37:55 -07:00
wuhang e6680f9e25 [Bugfix] Add log prefix in non-dp mode engine core (#21889) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-08-01 09:04:16 +00:00
Sungyoon Jeong 98df153abf [Frontend] Align tool_choice="required" behavior with OpenAI when tools is empty (#21052) 
Signed-off-by: Sungyoon Jeong <sungyoon.jeong@furiosa.ai>
 2025-08-01 07:54:17 +00:00
Songandoliveryuan 9484641616 [Model] Add step3 vl (#21998) 
Signed-off-by: oliveryuan <yuansong@step.ai>
Co-authored-by: oliveryuan <yuansong@step.ai>
 2025-07-31 23:19:06 +08:00
Sanchit Gandhiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ec02e536df [Bugfix] Relax lang pin for voxtral (#21833) 
Signed-off-by: Sanchit Gandhi <sgandhi3141@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-30 20:38:52 -07:00
Yan Pashkovsky bf668b5bf5 [Feature] Support multiple api keys in server (#18548) 
Signed-off-by: Yan Pashkovsky <yanp.bugz@gmail.com>
 2025-07-30 07:03:23 -07:00
rongfu.leng da3e0bd6e5 [Bugfix] we should use metavar is not choices (#21902) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-07-30 06:51:58 -07:00
Nick Hill 7234fe2685 [Misc] Rework process titles (#21780) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-29 05:14:47 +00:00
Keyang Ruandkeru 9ace2eaf35 [Bugfix] Improve JSON extraction in LlamaToolParser (#19024) 
Signed-off-by: keru <keyang.ru@oracle.com>
Co-authored-by: keru <keyang.ru@oracle.com>
 2025-07-28 12:36:58 +00:00
rongfu.leng 2cc571199b [feature] add log non default args in LLM (#21680) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-07-28 02:21:22 -07:00
Hongsheng Liuandwangzi 7656cf4cf3 [Bugfix] [issue-21565] Fix the incompatibility issue with stream and named function calling when Thinking is disabled (#21573) 
Signed-off-by: wangzi <3220100013@zju.edu.cn>
Co-authored-by: wangzi <3220100013@zju.edu.cn>
 2025-07-27 22:43:50 -07:00
Yuxuan ZhangandChenhui Zhang 93269bb43e Fix GLM tool parser (#21668) 
Co-authored-by: Chenhui Zhang <zhang.chenhui@outlook.com>
 2025-07-28 10:46:38 +08:00
Cyrus LeungandHarry Mellor 86ae693f20 [Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00
Alexandre JUAN 2f6e6b33fb [Bugfix] Fix isinstance check for tensor types in _load_prompt_embeds to use dtype comparison (#21612) 
Signed-off-by: Alexandre Juan <a.juan@netheos.net>
 2025-07-25 20:11:10 -07:00
mgazzandCyrus Leung e189b50f53 Add support for Prithvi in Online serving mode (#21518) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-07-25 07:01:27 -07:00
kourosh hakhamaneshi 9fe98d4250 [Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2025-07-25 06:49:11 -07:00
Cyrus Leung 46d81d6951 [V1] Get supported tasks from model runner instead of model config (#21585) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-25 05:36:45 -07:00
Cyrus Leung 34ddcf9ff4 [Frontend] run-batch supports V1 (#21541) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-24 20:05:55 -07:00
Chauncey 6da0078523 [Feat] Allow custom naming of vLLM processes (#21445) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-24 03:15:23 -07:00
Shintarou OkadaandHarry Mellor 6eca337ce0 Replace --expand-tools-even-if-tool-choice-none with --exclude-tools-when-tool-choice-none for v0.10.0 (#20544) 
Signed-off-by: okada <kokuzen@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 02:56:36 -07:00
Yuxuan Zhang 85bda9e7d0 remove GLM-4.5 quantization wrong Code (#21435) 2025-07-24 01:52:43 -07:00
Robert ShawRobert ShawNick HillTyler Michael Smith
 		d5b981f8b1 [DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-23 20:57:32 -07:00
deven-labovitch 63d92abb7c [Frontend] Set MAX_AUDIO_CLIP_FILESIZE_MB via env var instead of hardcoding (#21374) 
Signed-off-by: Deven Labovitch <deven@videa.ai>
 2025-07-23 20:22:19 -07:00