iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
17,474 Commits 3 Branches 0 Tags
16282a9c4ee754bedd67f72b01ac76ae8568dbdd
Commit Graph
1597 Commits
Author SHA1 Message Date
yzong-rhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6ec7dcd641 [Frontend][Metrics] Add vllm:tool_call_parser_invocations_total Prometheus metric (#44448) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-10 10:29:11 -04:00
Varun Vinayak ShenoyandCodex 5828a205ef Fix Harmony tool descriptions for optional fields (#44686) 
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
Co-authored-by: Codex <codex@openai.com>
 2026-06-09 23:29:22 -07:00
Flora Feng 6aec99f030 [Refactor] Remove dead states from chat completion serving (#45081) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-09 22:20:15 -07:00
Flora Feng d82ac00923 [Refactor][Mistral] Extract parsing logic into MistralParser (#44596) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-10 00:12:23 +00:00
Juan Pérez de Algaba 1b1359c332 [Security] Fix DoS via audio decompression bomb in speech-to-text endpoint (#44970) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-10 00:18:53 +08:00
Andreas Karatzas f0f6805d8a [CI] Stabilize the multi-audio OpenAI server path (#44051) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-07 15:54:32 +08:00
yzong-rh 703fb17b13 [Bugfix] GPT-OSS instruction rendering (#44330) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-05 13:52:32 -04:00
Vic Wen ef3af56a97 Fix LLM.wait_for_completion output type docstring (#44617) 
Signed-off-by: viiccwen <viiccwen@gmail.com>
 2026-06-05 00:16:38 -07:00
Wentao Ye 41a4829f22 [Logs Refactor] Optimize shutdown logs, easier to follow and consistent (#43707) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-04 14:36:32 -04:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-04 06:49:31 +00:00
Ted Mostly 0c1e6f63f5 [Bugfix] Fix VLLMNotFoundError when using LoRA adapter name in poolin… (#44410) 
Signed-off-by: Ted Mostly <wanghenshui@qq.com>
 2026-06-04 02:22:03 +00:00
Jee Jee Li 4d1fd13613 [CI/Build] Fix LoRA testing (#44425) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-06-03 08:58:06 -07:00
Flora Feng 209709a8c1 [Bugfix] Fix unstreamed tool call args dropped in Responses API streaming (#44348) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-03 03:19:08 -07:00
William Romandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f0204358d9 [Bugfix] fix crash in postprocess for null tool args (#43862) 
Signed-off-by: William-Rom <william.rom@intility.no>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-02 22:17:26 -07:00
ChaunceyAleksandar YanakievAng Kah Min, Kelvin <syraxius@hotmail.com>
 		ed9a7526b6 [Anthropic] Support system role messages inside messages array (#44283) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com>
Co-authored-by: Ang Kah Min, Kelvin <syraxius@hotmail.com>
 2026-06-02 18:13:54 +00:00
wang.yuqi b623f7ea95 [Frontend] Consolidate dev entrypoints. (#44170) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-02 06:30:21 -07:00
Isotr0py f8e9c56d15 [Multimodal] Automatically select registered video loader for VLM (#44126) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-02 09:09:47 +00:00
alberto e30313220c [Parser] Migrate ResponsesParser to unified Parser interface (#42977) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-06-02 08:50:05 +00:00
Flora Feng 68dafcca75 [Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (#44267) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 15:11:42 +08:00
Andreas Karatzas 54d0c36fff [CI] Stabilize OpenAI schema fuzzing for malformed structural tags (#44131) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-01 19:56:15 -07:00
Flora Feng 9affc17a05 [Refactor] Move unstreamed tool-arg flush from serving layer to parser (#44017) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 10:37:43 +08:00
Siddharth BedekarandOpenAI Codex 266b9d9c64 [Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096) 
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-01 15:37:30 -04:00
wang.yuqi 0910f7e0e1 [Frontend] Resettle generative scoring entrypoint. (#44153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-01 07:54:59 +00:00
Umut Polat f46e6be169 [Misc] Use VLLMValidationError consistently in chat completion and completion protocol validators (#36254) 
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
 2026-06-01 04:04:11 +00:00
yzong-rh 46409fd2a1 [Fronten] Clean up stop_token_ids override for Harmony (#44009) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-05-29 13:28:06 -07:00
Wentao Ye 5dbf1605a0 [Feature] SSL support for dp supervisor (#43688) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-29 19:28:12 +00:00
Chauncey 87f12e5c7c [Frontend]Responses API supports chat_template_kwargs (#43761) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-29 07:58:19 +00:00
Vadim Gimpelson 5d126dd155 [Bugfix] Exclude Ray DP from #42585's deferred port allocation (#43864) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-05-28 15:55:14 +00:00
Chauncey d692b89c2c [Feature] Add structured output and effort support to Anthropic Messages API (#42396) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-28 12:06:48 +00:00
Ignacio Sicaandsfeng33 7909f82a45 [Bugfix][Frontend] streaming tool-call serializer drops first args chunk when name and args share a DeltaMessage (#42683) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-28 05:20:55 +00:00
Ashwin GiridharanandChauncey 52a31ccecc [Bugfix] Map reasoning_effort to enable_thinking in chat template kwargs (#43401) 
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-27 05:39:49 -07:00
Vadim Gimpelsonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 812e7e7364 [Bugfix][V1] Fix TOCTOU race causing intermittent EADDRINUSE on multi-API-server DP startup (#42585) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-26 14:06:00 -07:00
linzm1007andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 739af5c7e1 [Reasoning] [Bugfix] Reject invalid thinking_token_budget values (#43402) 
Signed-off-by: linzm1007 <linzm1007@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-26 03:37:30 -07:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d5cf7b4a2c [Frontend] Split the offline inference APIs and utils. (#43553) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-26 05:20:24 +00:00
Andreas Karatzas 5940590855 [ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-24 10:06:49 +00:00
ffggsandCyrus Leung d8b385b7ea [Bugfix][Frontend] Fix input_audio parsing when uuid is present (#43414) 
Signed-off-by: ffggs <314137448@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-05-23 09:03:19 -07:00
Russell BryantandClaude Opus 4.6 2b94d1c0ca [Frontend] Simplify AuthenticationMiddleware path extraction (#43426) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-22 11:59:14 -07:00
Wentao Yerobertgshaw2-redhatRobert ShawNick Hill
 		0ddd7dd656 [Frontend] DP Supervisor (#40841) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: robertgshaw2-redhat <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-21 20:33:16 -07:00
ruizhangandRui Zhang 60af5c16ee [Frontend] Add truncation side to OpenAI endpoints (#43260) 
Signed-off-by: Rui Zhang <rza21.bc@gmail.com>
Signed-off-by: Rui Zhang <rui.zhang@globalrelay.net>
Co-authored-by: Rui Zhang <rui.zhang@globalrelay.net>
 2026-05-21 20:32:31 -07:00
Daoyuan Li a6682d1d25 [Bugfix] Warn when renderer_num_workers has no effect on offline LLM (#42905) 
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
 2026-05-20 21:35:08 -07:00
Nick HillandBugen Zhao f2ace1d57d [Frontend][RFC] Rust front-end integration (#40848) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-05-21 12:24:48 +08:00
Ben Browningandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 346cf163a1 [Frontend] Normalize reasoning_content to reasoning for client compatibility (#42664) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-20 21:23:47 -07:00
Aaron HaoandIsotr0py 2d6b3489b9 [R3] Add routed experts to openai entrypoint (#38939) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-20 09:07:59 -07:00
halleriteandClaude Opus 4.7 cb600d1cdb [Frontend] Forward X-data-parallel-rank header on /inference/v1/generate (#42330) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 08:58:46 +00:00
pmaybankgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fadf5d332c add enqueue all option to throughput benchmark (#42975) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 20:16:02 -07:00
Sage a78b842d0e [Bugfix] Fix top logprobs token placeholders in /inference/v1/generate (#42887) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-05-19 10:21:49 +00:00
wang.yuqi 301d986473 [Frontend] Consolidate beam search by BeamSearchMixin. (#42946) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-19 07:37:40 +00:00
Michael GoinandClaude 239b5ff30c [Frontend] Add --spec-method/--spec-model/--spec-tokens CLI aliases (#42476) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-18 17:22:27 -07:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00