iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
17,474 Commits 3 Branches 0 Tags
16282a9c4ee754bedd67f72b01ac76ae8568dbdd
Commit Graph
1107 Commits
Author SHA1 Message Date
yzong-rhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6ec7dcd641 [Frontend][Metrics] Add vllm:tool_call_parser_invocations_total Prometheus metric (#44448) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-10 10:29:11 -04:00
Varun Vinayak ShenoyandCodex 5828a205ef Fix Harmony tool descriptions for optional fields (#44686) 
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
Co-authored-by: Codex <codex@openai.com>
 2026-06-09 23:29:22 -07:00
Flora Feng 6aec99f030 [Refactor] Remove dead states from chat completion serving (#45081) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-09 22:20:15 -07:00
Flora Feng d82ac00923 [Refactor][Mistral] Extract parsing logic into MistralParser (#44596) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-10 00:12:23 +00:00
Andreas Karatzas f0f6805d8a [CI] Stabilize the multi-audio OpenAI server path (#44051) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-07 15:54:32 +08:00
yzong-rh 703fb17b13 [Bugfix] GPT-OSS instruction rendering (#44330) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-05 13:52:32 -04:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-04 06:49:31 +00:00
Jee Jee Li 4d1fd13613 [CI/Build] Fix LoRA testing (#44425) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-06-03 08:58:06 -07:00
Flora Feng 209709a8c1 [Bugfix] Fix unstreamed tool call args dropped in Responses API streaming (#44348) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-03 03:19:08 -07:00
wang.yuqi b623f7ea95 [Frontend] Consolidate dev entrypoints. (#44170) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-02 06:30:21 -07:00
alberto e30313220c [Parser] Migrate ResponsesParser to unified Parser interface (#42977) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-06-02 08:50:05 +00:00
Flora Feng 68dafcca75 [Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (#44267) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 15:11:42 +08:00
Andreas Karatzas 54d0c36fff [CI] Stabilize OpenAI schema fuzzing for malformed structural tags (#44131) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-01 19:56:15 -07:00
Flora Feng 9affc17a05 [Refactor] Move unstreamed tool-arg flush from serving layer to parser (#44017) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 10:37:43 +08:00
wang.yuqi 0910f7e0e1 [Frontend] Resettle generative scoring entrypoint. (#44153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-01 07:54:59 +00:00
Umut Polat f46e6be169 [Misc] Use VLLMValidationError consistently in chat completion and completion protocol validators (#36254) 
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
 2026-06-01 04:04:11 +00:00
yzong-rh 46409fd2a1 [Fronten] Clean up stop_token_ids override for Harmony (#44009) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-05-29 13:28:06 -07:00
Wentao Ye 5dbf1605a0 [Feature] SSL support for dp supervisor (#43688) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-29 19:28:12 +00:00
Chauncey 87f12e5c7c [Frontend]Responses API supports chat_template_kwargs (#43761) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-29 07:58:19 +00:00
Ignacio Sicaandsfeng33 7909f82a45 [Bugfix][Frontend] streaming tool-call serializer drops first args chunk when name and args share a DeltaMessage (#42683) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-28 05:20:55 +00:00
Ashwin GiridharanandChauncey 52a31ccecc [Bugfix] Map reasoning_effort to enable_thinking in chat template kwargs (#43401) 
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-27 05:39:49 -07:00
linzm1007andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 739af5c7e1 [Reasoning] [Bugfix] Reject invalid thinking_token_budget values (#43402) 
Signed-off-by: linzm1007 <linzm1007@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-26 03:37:30 -07:00
Andreas Karatzas 5940590855 [ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-24 10:06:49 +00:00
Russell BryantandClaude Opus 4.6 2b94d1c0ca [Frontend] Simplify AuthenticationMiddleware path extraction (#43426) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-22 11:59:14 -07:00
Wentao Yerobertgshaw2-redhatRobert ShawNick Hill
 		0ddd7dd656 [Frontend] DP Supervisor (#40841) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: robertgshaw2-redhat <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-21 20:33:16 -07:00
ruizhangandRui Zhang 60af5c16ee [Frontend] Add truncation side to OpenAI endpoints (#43260) 
Signed-off-by: Rui Zhang <rza21.bc@gmail.com>
Signed-off-by: Rui Zhang <rui.zhang@globalrelay.net>
Co-authored-by: Rui Zhang <rui.zhang@globalrelay.net>
 2026-05-21 20:32:31 -07:00
Nick HillandBugen Zhao f2ace1d57d [Frontend][RFC] Rust front-end integration (#40848) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-05-21 12:24:48 +08:00
Ben Browningandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 346cf163a1 [Frontend] Normalize reasoning_content to reasoning for client compatibility (#42664) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-20 21:23:47 -07:00
Aaron HaoandIsotr0py 2d6b3489b9 [R3] Add routed experts to openai entrypoint (#38939) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-20 09:07:59 -07:00
wang.yuqi 301d986473 [Frontend] Consolidate beam search by BeamSearchMixin. (#42946) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-19 07:37:40 +00:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
aoshen02 8c79ad6580 Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-05-13 23:49:01 -07:00
Walter Beller-Morales 873910d608 [Frontend] add support for thinking_token_budget in completions (#42116) 2026-05-13 16:01:52 -04:00
Ben Browning 0f69128a37 [Bugfix] Handle real-world gpt-oss tool call output in Harmony parsing (#42454) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-05-13 17:54:46 +00:00
yzong-rh 6ff7405b81 [Bugfix] [Frontend] Responses API, fix merging of messages (#42189) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Yifan <yzong@redhat.com>
 2026-05-12 16:09:59 +00:00
chazenandChauncey 7f65f84428 [Bugfix] Fix empty channel/recipient in harmony for /v1/responses (#35540) 
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com>
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-12 08:45:51 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d37e25ffbe [Frontend] Consolidate Speech to Text entrypoints. (#42370) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 07:06:57 +00:00
Chauncey 617239b70c [Frontend]Responses API supports chat_template_kwargs (#42272) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-11 11:59:39 +00:00
汪志鹏Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>CursorChauncey
 		581b5e9afc [Frontend] Return rendered prompt text in chat completion response (#42052) 
Signed-off-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Cursor <cursor@cursor.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-11 13:53:39 +08:00
abdulrahman-cohereCursor AgentNicolò Lucchesi
 		a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266) 
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com>
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-09 23:51:11 -07:00
Dao007forever 3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) 2026-05-09 22:52:09 -07:00
TomerBN-Nvidia 8189a15914 [Core] Replace routing replay with device cache and async D2H pipeline (#39917) 
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com>
 2026-05-07 11:24:56 -07:00
Flora Feng 8eb401134e [Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-07 09:50:59 -07:00
Viktor Pus d5b31c954d [Bugfix] Account for truncate_prompt_tokens when computing max_tokens (#41800) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
 2026-05-06 16:10:17 +00:00
jackQwertyJackchaunceyjiang
 		6467213a9f fix(openai): tolerate empty content in forced tool choice (#40148) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-06 07:16:03 -07:00
Chauncey c7aa186d67 [Frontend] Supports resubmitting output items with missing fields in Responses API (#41355) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-05 22:21:33 -04:00
Linzhang LiMichael GoinUbospicasfeng33
 		844df54269 feat: update xgrammar==0.2.0 to use structural tags for strict tool calling + reasoning for more models (#40894) 
Signed-off-by: Yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Ubospica <ubospica@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-04 12:45:24 -07:00
Bugen Zhaoandwang.yuqi a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-30 23:38:02 -07:00
Dhruv SingalClaude Opus 4.6Dhruv SingalYour NameOpenCodeSimon Mo
 		cb1b02d0e8 [Frontend] Add VLLM_SKIP_MODEL_NAME_VALIDATION environment variable (#34676) 
Signed-off-by: Dhruv Singal <dhruvsingalabc@gmail.com>
Signed-off-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Signed-off-by: Your Name <you@example.com>
Signed-off-by: vLLM Assistant <assistant@vllm.ai>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: OpenCode <noreply@openai.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-29 23:19:09 -07:00