iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,431 Commits 3 Branches 0 Tags
c1909e7e8ccd2037e76536a8e726120c85d3754e
Commit Graph
687 Commits
Author SHA1 Message Date
Kwai-Keye 8452946c06 [Model][VLM] Support Keye-VL-8B-Preview (#20126) 
Signed-off-by: Kwai-Keye <Keye@kuaishou.com>
 2025-07-01 23:35:04 -07:00
Chenheli Hua 2e7cbf2d7d [Frontend] Support configurable mm placeholder strings & flexible video sampling policies via CLI flags. (#20105) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-07-01 23:34:03 -07:00
Lifans 9ec1e3065a [Misc][Doc] Add missing comment for LLM (#20285) 
Signed-off-by: Lifan Shen <lifans@meta.com>
 2025-07-01 19:04:24 -07:00
Shintarou Okada 3d19d47d91 [Frontend] Expand tools even if tool_choice="none" (#17177) 
Signed-off-by: okada shintarou <okada@preferred.jp>
 2025-07-01 12:47:38 -04:00
Nicolò Lucchesi 314af8617c [Docs] Update transcriptions API to use openai client with stream=True (#20271) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-01 15:47:13 +00:00
Yuxuan ZhangandIsotr0py ed70f3c64f Add GLM4.1V model (Draft) (#19331) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-01 12:48:26 +00:00
fyuan1316 e28533a16f [Bugfix] Fix include prompt in stream response when echo=true (#15233) 
Signed-off-by: Yuan Fang <yuanfang@alauda.io>
 2025-07-01 01:30:14 +00:00
Kyle Sayers d8cf819a9a [Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
Nicolò Lucchesi daceac57c7 [Frontend] Generalize v1/audio/transcriptions endpoint (#20179) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-28 08:15:26 -07:00
Ilya Lavrenov aa0dc77ef5 [Perf] Improved perf for resolve_chat_template_content_format (#20065) 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@cerebras.net>
 2025-06-27 09:16:41 +00:00
Yazan Sharaya 6e244ae091 [Perf][Frontend] eliminate api_key and x_request_id headers middleware overhead (#19946) 
Signed-off-by: Yazan-Sharaya <yazan.sharaya.yes@gmail.com>
 2025-06-27 00:44:14 -04:00
Michael Goin 754b00edb3 [Bugfix] Fix Mistral tool-parser regex for nested JSON (#20093) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-26 01:01:17 +00:00
Nicolò LucchesiandRoger Wang e795d723ed [Frontend] Add /v1/audio/translations OpenAI API endpoint (#19615) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-06-25 17:54:14 +00:00
Max Wittig f59fc60fb3 [Feat][CLI] enforce-include-usage (#19695) 
Signed-off-by: Max Wittig <max.wittig@siemens.com>
 2025-06-25 01:43:04 -04:00
Nick Hill 8619e7158c [BugFix] Fix multi-node offline data parallel (#19937) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-24 12:45:20 -07:00
Aaron Pham c4cf260677 [Perf][CLI] Improve overall startup time (#19941) 2025-06-22 23:11:22 +00:00
Ye (Charlotte) Qi 2c11a29f0b [Misc] Simplify vllm bench cli subcommand implementation (#19948) 2025-06-22 12:34:48 -04:00
Rabin Adhikari 8ca81bb069 Fix: Check the type of params to be a Sequence not list. (#19910) 
Signed-off-by: Rabin Adhikari <rabin.adk1@gmail.com>
 2025-06-20 23:03:17 +00:00
Thomas Parnell 7771d1de88 [Fix] import regex instead of re (#19875) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-06-20 11:16:48 +00:00
xzbdmw ea10dd9d9e [Frontend] early return chat format resolution when specified (#19735) 2025-06-19 18:49:59 +00:00
Alex Brooks ead2110297 [Core][Bugfix] Fix Online MM Beam Search (#19688) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-06-19 17:18:07 +00:00
NekoMimiUnagiand22quinn 466166dcfd [Frontend] Add optional token-level progress bar to LLM.beam_search (#19301) 
Signed-off-by: Ruosen Li <rxl190028@utdallas.edu>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Ubuntu <ubuntu@ip-172-31-71-179.ec2.internal>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-19 03:21:41 -04:00
Zuxin 1d0ae26c85 Add xLAM tool parser support (#17148) 2025-06-19 14:26:41 +08:00
Maximilien de Bayserand22quinn 799397ee4f Support embedding models in V1 (#16188) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-18 21:36:33 -07:00
afeldman-nmandNick Hill dfada85eee [Frontend] Expose custom args in OpenAI APIs (#16862) 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-18 17:41:11 -07:00
nguyenhoangthuan99 ede5c4ebdf [Frontend] add chunking audio for > 30s audio (#19597) 
Signed-off-by: nguyenhoangthuan99 <thuanhppro12@gmail.com>
 2025-06-17 11:34:00 +08:00
Chauncey 836d4ce140 [Bugfix] fix missing 'finish_reason': null in streaming chat (#19662) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-16 14:10:39 +00:00
Francesco BertolottiFrancesco BertolottiAaron Pham
 		367871a469 [Misc][Frontend] passthrough bad_words (#19564) 
Signed-off-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai>
Co-authored-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
 2025-06-16 05:05:13 +00:00
maobaolong 08500011d3 [Fix] Convert kv_transfer_config from dict to KVTransferConfig (#19262) 2025-06-14 12:32:07 -07:00
Luka Govedič 3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Hyogeun Oh (오효근) bb4a0decef [Misc] Correct broken docs link (#19553) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-06-12 22:27:13 -07:00
wonjun Jang 1b0b065eb5 [BugFix] Handle missing sep_token for Qwen3-Reranker in Score API (#19522) 
Signed-off-by: strutive07 <strutive07@gmail.com>
 2025-06-12 07:00:47 +00:00
22quinn 7e3e74c97c [Frontend] Improve error message in tool_choice validation (#19239) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-12 01:13:00 -04:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Reidandreidliu41 6cd4ae8acd [Frontend] Add tqdm_leave_pbar to control progress bar visibility (#19357) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-10 04:55:09 +00:00
liusiqian-tal 31f58be96a [Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) 
Signed-off-by: liusiqian <liusiqian@tal.com>
 2025-06-09 21:41:21 +00:00
Kseniya Parkhamchuk 8335667c22 [Frontend] Remove unreachable code from llm.py (#19288) 
Signed-off-by: KsuParkhamchuk <k.parkhamchuk@gmail.com>
 2025-06-09 10:22:10 +08:00
Xu Song 3da2313d78 Support allowed_token_ids in ChatCompletionRequest (#19143) 
Signed-off-by: Xu Song <xusong.vip@gmail.com>
 2025-06-06 05:06:48 +00:00
Patrick von Platen f20f9f063b [mistral_common] Add v11 tokenizer (#19193) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-06-05 08:27:41 -07:00
Reidandreidliu41 1aeb925f34 [Frontend] improve vllm run-batch --help display (#19187) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-05 11:16:25 +00:00
Chauncey 8fc57501d3 [Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled (#19135) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-05 06:24:24 +00:00
Chauncey 4de790fcad [Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled (#19075) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-03 23:27:24 +00:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Rui Qiao bdce64f236 [V1] Support DP with Ray (#18779) 2025-06-02 21:15:13 -07:00
FrαnçoisandCyrus Leung 20133cfee2 [Frontend] enable custom logging for the uvicorn server (OpenAI API server) (#18403) 
Signed-off-by: François Paupier <francois.paupier@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-06-02 15:04:23 +00:00
Reidandreidliu41 20079c6e36 [Misc] add return token strs for tokenize (#18941) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-31 18:00:11 +00:00
Lucia FangandLucia Fang b8b904795d fix security issue of logging llm output (#18980) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-05-31 10:38:56 +00:00
Pooya Davoodi dff80b0e42 [Frontend] Add rerank support to run_batch endpoint (#16278) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
 2025-05-31 07:40:01 +00:00
Will Eaton 1dab4d5718 Tool parser regex timeout handling (#18960) 
Signed-off-by: Will Eaton <weaton@redhat.com>
 2025-05-30 21:02:54 +00:00
Nick Hill 2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00