iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1354 Commits
Author SHA1 Message Date
Harry Mellor b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor 1c0c68202c Fix per file ruff ignores related to typing (#26254) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 16:37:55 +00:00
Harry Mellor 4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung a964e5e6c3 [Bugfix] Allow --skip-tokenizer-init with echo and return_token_ids (#26238) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-05 05:38:53 +00:00
Cyrus Leung 119f00630b [Renderer] Clean up renderer code (#26216) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-04 17:05:29 +00:00
Isotr0py a42d2df75f [Frontend] Cache chat template kwargs resolution (#26227) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-04 15:32:30 +00:00
Ben BrowningandCyrus Leung ea25a76c05 [BugFix] Use async Mistral Tokenizer in Chat Completions (#26134) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-04 09:42:08 +08:00
Andrew XiaandAndrew Xia 831b124151 [responsesAPI] add better error messaging for long prompts (#25724) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-10-03 14:33:13 -07:00
Cyrus Leung d78fda7cda [Renderer] Move Processor out of LLMEngine (#26165) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 15:08:22 +00:00
Aleksandr Samarin 73a99cc2a5 [Model] Fixed stream generator for gpt-oss + spec-decoding (#26027) 
Signed-off-by: Aleksandr Samarin <astrlrd@nebius.com>
 2025-10-03 13:43:41 +00:00
Yang LiuandDarkLight1337 812b7f54a8 [Renderer] Move Processor out of AsyncLLM (#24138) 
Signed-off-by: Yang <lymailforjob@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 11:29:45 +00:00
kyt 2ed3f20dba [openai] Fix missing tool usage check (system message) (#24768) 
Signed-off-by: kyt <eluban4532@gmail.com>
 2025-10-03 18:55:44 +08:00
Andrew XiaandAndrew Xia e5017cd6d6 [gpt-oss] disable tool server initialization if no tool in request (#25790) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-10-03 05:08:35 +00:00
Andrew XiaandAndrew Xia 5db1870bb9 [gpt-oss] use vLLM instead of openai types for streaming (#25186) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-09-30 22:47:07 +00:00
Russell BryantandIsotr0py 7977e5027c Add filtering for chat template kwargs (#25794) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-27 10:46:49 +00:00
Russell Bryantandrentianyue-jk 3f5d902d2a Validate API tokens in constant time (#25781) 
Signed-off-by: rentianyue-jk <rentianyue-jk@360shuke.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: rentianyue-jk <rentianyue-jk@360shuke.com>
 2025-09-27 18:09:26 +08:00
Russell BryantandChenheli Hua 3958b96bf5 Add option to restrict media domains (#25783) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-27 01:23:52 +00:00
Xu Wenqing b03b1b97f6 Support LongCat-Flash-Chat tool call (#24083) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-09-26 09:25:39 +00:00
Harry MellorandCyrus Leung 70fbdb26e9 Add backward compatibility for guided_... API (#25615) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-25 19:45:25 +08:00
wang.yuqi 7f570f1caa [V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-25 11:26:31 +00:00
Harry Mellor e7f27ea648 Improve --help for enhanced user experience (#24903) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-24 23:08:18 +00:00
Tao Hui e18b714b2e [Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in non-streaming output (#25405) 
Signed-off-by: taohui <taohui3@gmail.com>
 2025-09-24 20:58:00 +08:00
Woosuk Kwon 2e19a848d4 [V0 Deprecation] Remove max_seq_len_to_capture (#25543) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-24 01:51:39 -07:00
Ben Browning 5caaeb714c [Bugfix] [Frontend] Cleanup gpt-oss non-streaming chat tool calls (#25514) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2025-09-24 03:20:38 +00:00
Andrew Xia 95bc60e4cb [gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-23 15:46:46 -07:00
Alec SandAlec Solder c828d1bf98 [Bugfix] gpt-oss container tool output bug (#25485) 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
 2025-09-23 20:43:45 +00:00
Zhikaiiii 9383cd6f10 [Frontend] Add a new xml-based tool parser for qwen3-coder (#25028) 
Signed-off-by: Zhikaiiii <1658973216@qq.com>
 2025-09-23 16:07:27 +08:00
Alec SAlec SolderYe Qi
 		45d7d852d3 [Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) 
Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-22 23:38:19 +00:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 3d2c56b7a9 Make mypy behave like a proper pre-commit hook (#25313) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-22 12:23:45 +00:00
WeiQing ChenwangziChauncey
 		0eecb31663 [Bugfix] Fix hermes tool parser handling of non-string argument types (#22002) 
Signed-off-by: wangzi <3220100013@zju.edu.cn>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: wangzi <3220100013@zju.edu.cn>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-09-22 11:35:39 +08:00
Woosuk Kwon 0ff8ebb2d7 [V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 08:52:32 -07:00
Cyrus Leung 65a5910ce3 [Optimization] Cache chat template result when processor fails to be loaded (#25341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-21 19:41:02 +08:00
Woosuk Kwon 52c2a8d4ad [V0 Deprecation] Remove LLMEngine (#25033) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-20 17:56:30 -07:00
Cyrus Leung 3d9a1d2de5 [V1] Support LLM.apply_model (#18465) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 07:14:35 +00:00
Chaunceyandxin.li f91480b2d4 [Bugfix] fix tool call arguments is empty (#25223) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: xin.li <xin.li@daocloud.io>
 2025-09-20 13:29:54 +08:00
Maximilien de Bayser a36c675817 Don't skip special tokens with hermes-style tool calling (#25281) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-09-19 17:33:25 -07:00
Alec SAlec SolderYe Qi
 		e69e0b8b5f [Frontend] Responses API messages out, just harmony for now (#24985) 
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-19 21:40:16 +00:00
Cyrus Leung 6c117cff7d [Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Harry Mellor 058525b997 Move PoolerConfig from config/__init__.py to config/pooler.py (#25181) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 11:02:55 +00:00
Andrew Xia 6d8246aaff [gpt-oss] Add ResponseReasoningPartAddedEvent, ResponseReasoningPartDoneEvent for streaming (#24938) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-18 19:11:59 -07:00
Andrew SansomandCyrus Leung 9a4600e4dc [CORE] Prompt Embeddings Support for v1 Engine (#24278) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-19 08:03:09 +08:00
Woosuk Kwon e19bce40a1 [V0 Deprecation] Remove AsyncLLMEngine (#25025) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-18 11:07:42 -07:00
Hyogeun Oh (오효근) b419937c78 [Docs] Fix warnings in mkdocs build (continued) (#25163) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-18 08:23:26 -07:00
wang.yuqiandIsotr0py 5f696c33b1 [New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-18 23:22:01 +08:00
dongbo910220andClaude 67244c86f0 feat(api): Return 503 on /health when engine is dead (#24897) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-09-18 14:29:40 +00:00
Chauncey cc935fdd7e [Frontend] Support setting logprobs to -1 (#25031) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-18 10:34:42 +00:00
Aaron PhamandHarry Mellor 29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Simon Mo e111d5b0ae [CLI] Use streaming in CLI chat and completion commands (#23769) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-09-17 22:30:26 -07:00
Andrew Sansom bec060fd99 Mark prompt logprobs as incompatible with prompt embeds at API level (#25077) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-17 21:25:07 -07:00