iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1354 Commits
Author SHA1 Message Date
Andrew Xia bff2e5f1d6 [gpt-oss][2] fix types for streaming (#24556) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-17 22:04:28 +00:00
Shijun Yin 2b85697031 [BugFix] enable DOTALL to match multi-line tool_call parameters in extract_tool_call_required_streaming (#24668) 
Signed-off-by: Shijun Yin <shijun.yin@outlook.com>
 2025-09-17 09:21:18 +00:00
Chauncey 544fe76b95 [Frontend] Support returning all prompt logprobs (#24956) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-17 09:03:52 +00:00
Zhuohan Li 6c47f6bfa4 [Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Woosuk Kwon 5801e49776 [V0 Deprecation] Remove MQLLMEngine (#25019) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-16 21:29:27 -07:00
Prashant Gupta ea3de5ef0d [misc] fix typo in value error (#24995) 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
 2025-09-16 20:58:38 -07:00
Andrew Xia 86daa875fe [gpt-oss][1][bugfix] fix streaming final output (#24466) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-16 13:56:16 -06:00
Andrew Xia f4d6eb95cf [gpt-oss][1b] streaming add item id, content id (#24788) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-16 18:41:12 +00:00
Cheng Kuan Yong JasonandChauncey 68dbde5dbb [Bugfix] remove duplicate tokens streamed in required tool choice streaming (#23312) 
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-09-16 15:16:32 +08:00
Andrew Xia 73df49ef3a [gpt-oss][1a] create_responses stream outputs BaseModel type, api server is SSE still (#24759) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-15 13:08:08 -07:00
Andrew Xia 25aba2b6a3 [gpt-oss] Add IncompleteDetails to ResponsesRepsonse (#24561) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-09-15 13:07:55 -07:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> c4afdb69cc Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
Chenheli HuaRoger WangRoger Wang
 		7f2ea7074e [Frontend][Multimodal] Allow skipping media data when UUIDs are provided. (#23950) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-09-13 02:16:06 +00:00
Kebe 684b6870e1 [Bugfix][Frontend] Fix --enable-log-outputs does not match the documentation (#24626) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-12 08:01:24 -07:00
Didier Durand bcb06d7baf [Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
Maximilien de Bayser e090b7b45b Enable conversion of multimodal models to pooling tasks (#24451) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-09-12 03:30:41 +00:00
Nick Hill b971f91504 [BugFix] Fix tokenize asyncio task leak (#24677) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-11 19:44:04 +00:00
Harry Mellor c1eda615ba Fix model name included in responses (#24663) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:47:51 -07:00
Boyuan FengandHarry Mellor 94e6b2d55f Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Didier Durand e2b1f863aa [Doc]: fixing doc typos (#24635) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-10 23:19:28 -07:00
rongfu.lengandCyrus Leung 09e68bce34 [Misc] update log level debug to warning when process port is used by (#24226) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-10 11:32:57 -07:00
co63ocandRussell Bryant 3144d90217 fix some typos (#24167) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-09-10 06:21:23 -07:00
Flora Feng 77f62613f9 Consolidate rendering parameters into RenderConfig dataclass (#24543) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 08:44:47 +00:00
Flora Feng 15cb047e25 Extend renderer with embedding support and integrate completion endpoint (#24405) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 01:46:46 +08:00
Chen Zhang 1116590b16 [gpt-oss] Validate gpt-oss python tool during initialization (#23856) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-09 08:37:48 +00:00
Harry Mellor 3e0d4a3475 Move KVTransferConfig from config/__init__.py to config/kv_transfer.py (#24434) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-08 20:30:32 -07:00
cjackal 13b89bd823 [doc] update vllm serve cli args documentation (#24329) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-09-09 03:07:58 +00:00
zhiweizzhiweizAaron PhamSimon MoLu Fang
 		170129eb28 [gpt-oss] Harmony changes with container tool support (#23386) 
Signed-off-by: zhiweiz <zhiweiz@fb.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: zhiweiz <zhiweiz@fb.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2025-09-08 19:03:50 -07:00
Chenheli HuaRoger WangRoger WangCyrus Leung
 		01dfb5e982 [Frontend] User-provided uuids for medias in chat. (RFC #22044) (#23449) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-08 06:42:20 -07:00
Didier Durand f4962a6d55 [Doc]: fix typos in Python comments (#24417) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-08 00:22:16 -07:00
Chauncey 425b04b8f4 [gpt-oss][Responses API] Fix the function call id format (#24409) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-08 06:49:52 +00:00
Flora Feng 0661cb9df3 Add renderer-based prompt processing for embedding and classification endpoints (#24356) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-07 08:26:48 +00:00
Ye (Charlotte) Qi a3645ed94d [Frontend][Responses API] Support reporting tool output tokens and fix reasoning token count (#24285) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-06 13:27:15 -07:00
Aaron Pham fb691ee4e7 [Fix] [gpt-oss] fix non-tool calling path for chat completion (#24324) 2025-09-06 19:10:32 +00:00
Didier DurandandWentao Ye 35bf193864 [Doc]: fix typos in Python comments (#24294) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-05 19:41:12 -07:00
Chauncey 23a6c5280e [gpt-oss][Bugfix]Fix streamableparser for missing handling of certain token_ids (#24306) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-05 10:26:00 -07:00
Aaron PhamandSimon Mo c29fb540ff [gpt-oss] tool parser supports for /chat/completions [1/n] (#22386) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-09-04 20:39:12 -07:00
Nicolò Lucchesi 65e038931d [Frontend] Skip unnecessary detokenization when token_id is requested (#24236) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-04 23:04:12 +00:00
Kebe 8f423e5f43 [Feature][Response API] Add streaming support for non-harmony (#23741) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-04 17:49:06 +08:00
Flora Feng 712b273f65 [Refactor] Introduce basic Renderer for completion-style request (#24010) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-04 05:21:12 +00:00
wuhang a38f8bd54c [Feature][Responses API]Support MCP tools with streaming mode + background mode (#23927) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-09-04 04:05:10 +00:00
George Nagy II 36c260dad6 [Feature][gpt-oss] Add support for num_cached_tokens and num_reasoning_tokens tracking (#23460) 
Signed-off-by: George Nagy II <george.nagy0969@gmail.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-03 21:08:47 +00:00
Jakub Smid 28f350e147 Support add_generation_prompt in embeddings endpoint with chat request (#23931) 
Signed-off-by: biba10 <jaksmid@seznam.cz>
 2025-09-03 10:47:55 +00:00
co63oc 1bd007f234 fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Chenheli Hua f399182e8c Run ruff format on a few files. (#24075) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-02 17:55:32 +00:00
Didier DurandandJee Jee Li 0235103cbb [Doc]: fix typos in Python comments (#24042) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-01 19:07:45 -07:00
Woosuk Kwon 5685370271 [Chore][V0 Deprecation] Move LogProb to a separate file (#24055) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-01 12:07:53 -07:00
Christian Pinto cf91a89dd2 [docs][misc] IOProcessor plugins fixes (#24046) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2025-09-01 09:17:41 -07:00
Nicolò LucchesiandCyrus Leung d46934b229 [Frontend] Gemma3n audio transcriptions/translations endpoint (#23735) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-01 18:07:46 +08:00
Didier Durand 107284959a [Doc]: fix typos in Python comments (#24026) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-01 09:38:20 +00:00