iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
16,919 Commits 3 Branches 0 Tags
1806d1adfc9b598bc6eb94de38a330aaad04c291
Commit Graph
66 Commits
Author SHA1 Message Date
Andreas Karatzas 5940590855 [ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-24 10:06:49 +00:00
halleriteandClaude Opus 4.7 cb600d1cdb [Frontend] Forward X-data-parallel-rank header on /inference/v1/generate (#42330) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 08:58:46 +00:00
Sage a78b842d0e [Bugfix] Fix top logprobs token placeholders in /inference/v1/generate (#42887) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-05-19 10:21:49 +00:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Hongxin XuClaude Opus 4.6arlenxuJunjie Zhang
 		c7560af424 [RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support (#39568) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
 2026-05-14 14:12:30 +00:00
halleriteandClaude Opus 4.7 97c4317bf5 [Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate (#42329) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 11:16:46 +02:00
Sumanth R Hegde e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Viktor Pus d5b31c954d [Bugfix] Account for truncate_prompt_tokens when computing max_tokens (#41800) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
 2026-05-06 16:10:17 +00:00
Luis 🚀andAndrew Sansom 14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
Chenguang ZhengandOpenAI Codex 07351e0883 [Feature] Warm up readonly multimodal processor during renderer startup (#40797) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-25 03:57:41 +00:00
Neil Schemenauer 56bdf85e10 [Feature] Avoid eager import of the "mistral_common" package. (#40043) 
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
 2026-04-24 02:49:16 +00:00
Bugen Zhao 0210024ae7 [Bugfix] Pass effective chat template kwargs to reasoning parsers (#40460) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-04-21 22:17:51 -07:00
Nithin ChalapathiandCyrus Leung 80b18230e0 [Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405) 
Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com>
Signed-off-by: Nithin Chalapathi <nithinc@berkeley.edu>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-04-17 20:31:56 -07:00
Julien Denize c0722f22de [Mistral Grammar] Fix tool and reasoning parsing (#39217) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
 2026-04-15 21:05:04 -07:00
rishitdholakia13mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Aaron Pham
 		70ed01550c [Reasoning][Frontend] Add model config to adjust_request in reasoning parser (#37848) 
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Signed-off-by: rishitdholakia13 <123388671+rishitdholakia13@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2026-04-14 20:29:06 +00:00
wang.yuqi 66c079ae83 [Frontend][4/n] Improve pooling entrypoints | pooling. (#39153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-09 10:09:45 +00:00
Ben BrowningClaude Opus 4.6Cursor
 		8477fe427d [Tool] adjust_request to reasoning parser, and Gemma4 fixes (#39027) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-04-08 19:04:04 +00:00
Hyeonki Hong 25f2b55319 [Frontend] feat: add streaming support for token generation endpoint (#37171) 
Signed-off-by: Hyeonki Hong <hyeonki.hong@moreh.io>
 2026-04-03 10:20:32 +00:00
Sergey Zinchenko 5a2d420c17 [Bugfix] Use dedicated MM processor cache in /tokenize to prevent sender-cache pollution (#38545) 
Signed-off-by: Sergey Zinchenko <sergey.zinchenko.rnd@gmail.com>
 2026-04-01 21:14:49 -07:00
Flora Feng 3e802e8786 [Mypy] Fix adjust_request typing (#38264) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-31 04:21:18 +00:00
Flora Feng e2db2b4234 [Tool Parser][1/3] Pass tools to ToolParser constructor (#38029) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-26 10:29:06 +08:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Flora Feng 2e67fa756d Fix tool_parser_cls type annotation from Callable to type[ToolParser] (#37957) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-23 22:58:27 -07:00
Sage 59192dfd39 [Frontend] Complete OpenAI render delegation (#37287) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-17 13:53:55 +00:00
Umut Polat 56cb1baa66 [Misc] Use VLLMValidationError in batch, pooling, and tokenize protocol validators (#36256) 
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
 2026-03-17 13:52:30 +00:00
Sage 00f8e0d211 [Frontend] Delegate tokenization serving preprocessing to OpenAIServingRender (#37266) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-17 11:22:54 +00:00
Roy Wang 0115e957d4 [Frontend][Misc] Remove unused log in /is_sleeping (#37093) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
 2026-03-16 17:46:28 +08:00
Andreas Karatzas 57a314d155 [CI][Bugfix] Fix 500 errors from priority overflow and TemplateError subclasses in schema fuzz tests (#37127) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-16 05:27:21 +00:00
Isotr0py 143e4dccdf [Misc] Add online audio_in_video test (#36775) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-15 00:14:11 -07:00
Sergey Zinchenko 4a718e770d [Bug] Fix Failure in /v1/chat/completions/render for Multimodal Requests (https://github.com/vllm-project/vllm/issues/35665) (#35684) 2026-03-14 14:10:11 +00:00
Sage a2268617cf [Frontend] Delegate preprocessing to OpenAIServingRender (#36483) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-13 00:39:43 -07:00
Sage 06e0bc21d2 [Frontend] Split OpenAIServingModels into OpenAIModelRegistry + OpenAIServingModels (#36536) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-12 03:29:37 -07:00
Julien Denize a3ea760ea5 Add 'none' reasoning effort to ChatCompletionRequest (#36238) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2026-03-11 15:45:34 +00:00
Ning Xie 40c0461f24 [openapi] refactor render related openapi [3/N] (#36749) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-03-11 03:14:34 -07:00
Ning Xie fe714dd507 [openapi server] log exception in exception handler(2/N) (#36201) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-03-10 20:16:30 -07:00
Sage 4497431df6 [Frontend] Add GPU-less render serving path (vllm launch render) (#36166) 2026-03-08 16:35:09 +01:00
milesial 755356b3d1 feat: expose media_io_kwargs at runtime (#34778) 
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
 2026-03-07 04:27:04 +00:00
Ning Xie 176c799f4c [openai api] log exception in exception handler (1/N) (#31164) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-03-05 16:00:12 +00:00
Nick Hill dbf0da817a [Core] Cleanup engine pause/sleep logic (#34528) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-24 19:33:34 -08:00
Cyrus Leung 574fe75245 [Renderer] Move InputPreprocessor into Renderer (2/2) (#34560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-17 05:29:01 -08:00
Stanislav KirillovStanislav KirillovCyrus Leung
 		50dbd6c9e6 [bugfix] Fix critical bug when reporting for all paths where handler.create_error_response is used (#34516) 
Signed-off-by: Stanislav Kirillov <stas@nebius.com>
Co-authored-by: Stanislav Kirillov <stas@nebius.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-14 23:24:25 -08:00
Andreas Karatzas 98bcc6ca59 [CI][Entrypoints] Validate detokenize token IDs to prevent int64 overflow causing 500 (#34468) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-14 23:08:38 -08:00
Cyrus Leung b96f7314b4 [Refactor] Pass Renderer to Input Processor (#34329) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-11 19:38:11 -08:00
Reagan LeeandReagan Lee fdd6f2ad58 Convert online APIs to use Renderer (#34084) 
Signed-off-by: Reagan Lee <“reaganjlee@gmail.com”>
Co-authored-by: Reagan Lee <“reaganjlee@gmail.com”>
 2026-02-10 19:44:31 +00:00
wang.yuqi dab1de9f38 [Frontend][CI] Consolidate instrumentator entrypoints (#34123) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-10 07:30:19 +00:00
Aaron HaoandRobert Shaw 89a385d79f [Feat][RL] Pause and Resume with keep requests for single engine (#32351) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-07 00:08:58 +00:00
Sumanth R Hegde ae2e93f89b [Fix] Fix logprobs=0 handling for /inference/v1/generate endpoint (#34010) 
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-06 20:33:40 +00:00
Aaron HaoandSumanthRH c1858b7ec8 [Feat][RL][1/2] Native Weight Syncing API: NCCL (#31943) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-05 12:13:23 -05:00
Xiao Yang 2238a12c13 [Misc] support collect_env for endpoint /server_info (#33246) 
Signed-off-by: yang.xiao <yang.xiao@daocloud.io>
 2026-02-01 01:42:59 +08:00
jma99_2333andRoger Wang 22d9a056d5 Support clear mm and encoder cache (#33452) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-31 15:22:25 +00:00