iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,776 Commits 3 Branches 0 Tags
94de871546e8da687c08ed8a7e0a26531500d4bd
Commit Graph
922 Commits
Author SHA1 Message Date
wang.yuqi dab1de9f38 [Frontend][CI] Consolidate instrumentator entrypoints (#34123) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-10 07:30:19 +00:00
Andrew XiaandAndrew Xia 9608844f96 [responsesAPI] fix simpleContext streaming output_messages (#34188) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-02-09 22:53:07 -08:00
kourosh hakhamaneshiandCyrus Leung a75a5b54c7 [bug-fix] supported_tasks is breaking backward compatibility at init_app_state (#34027) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Signed-off-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-09 09:46:46 +08:00
Cyrus Leung b956cdf818 [Doc] Fix run_batch docs (#34056) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 06:18:16 -08:00
Pooya DavoodiandCyrus Leung 2cb2340f7a [Frontend]Add support for transcriptions and translations to run_batch (#33934) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-07 05:24:57 -08:00
Cyrus Leung edb359cce4 [Renderer] Define render_cmpl and render_chat (#34039) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 05:24:40 -08:00
Cyrus Leung 11a4c9d30d [Misc] Simplify get_max_tokens (#34036) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 00:59:49 -08:00
kourosh hakhamaneshiandCursor 1c3b22058f [Misc] Add backward-compatible import aliases for renamed translations module (#34015) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-07 11:01:41 +08:00
Cyrus Leung cd8b405bd0 [Refactor] Consolidate sequence normalization and enc-dec parsing (#33928) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-06 15:43:47 +00:00
emricksini-h 325ab6b0a8 [Feature] OTEL tracing during loading (#31162) 2026-02-05 16:59:28 -08:00
Nicolò Lucchesi 20f5d185a6 [Misc] Rename translations to speech_to_text for OAI serving component (#33904) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-05 19:16:52 +00:00
Chauncey 6abb0454ad [Perf] Optimize the performance of structured output + reasoning (#33557) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-02-05 15:45:29 +08:00
Andrew XiaandAndrew Xia 9595afda18 [2/N] move responses/serving _make_response_output_items logic to parser (#33281) 
Signed-off-by: Andrew Xia <axia@fb.com>
Signed-off-by: Andrew Xia <axia@meta.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-02-05 13:46:15 +08:00
Andreas Karatzas fb1270f1f8 [CI][Bugfix]: return McpCall for built-in MCP tools in non-streaming mode (#32762) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-05 11:14:06 +08:00
Chauncey f67ee8b859 [Perf] Optimize chat completion streaming performance (#33782) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-02-04 12:30:36 +00:00
Andrew XiaandAndrew Xia e1bf04b6c2 [1/N] Initial Implementation of Parser for ResponsesAPI (#32712) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-02-04 10:59:03 +08:00
Harry Mellor 61e632aea1 Turn @config into a dataclass_transform (#31541) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-03 17:40:59 +00:00
Patrick von PlatenandCyrus Leung f0d5251715 [Voxtral models] Skip warm-up to skip confusing error message in warm-up (#33576) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-03 07:22:34 -08:00
Daniel MeschederandDaniel Mescheder 4c4b6f7a97 [Frontend] Add sampling parameters to Responses API (#32609) 
Signed-off-by: Daniel Mescheder <dmesch@amazon.com>
Co-authored-by: Daniel Mescheder <dmesch@amazon.com>
 2026-02-03 13:51:10 +08:00
Cyrus Leung b10d05b8a8 [Model] Use explicit types in get_generation_prompt (#33551) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-02 12:38:49 +00:00
Cyrus Leung a502831d36 [Chore] Remove redundant input parsing methods (#33542) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-02 10:50:47 +00:00
Cyrus Leung 79b6ec6aab [Bugfix] Fix inconsistent handling of cache reset (#33481) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 20:23:41 -08:00
cmunley1 6f5e7cda57 support return prompt token ids in responses (#33378) 2026-01-31 06:04:20 -08:00
Cyrus Leung f0a1c8453a [Frontend] Use new Renderer for Completions and Tokenize API (#32863) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 04:51:15 -08:00
Nicolò Lucchesi e77f162cf5 [Bugfix] Fix Qwen3ASR language asr tag in output (#33410) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-31 05:24:49 +00:00
Harry Mellor c5113f60f2 Remove deprecated reasoning_content message field (#33402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-30 11:48:15 +00:00
Patrick von PlatenandNick Hill 10152d2194 [Realtime API] Adds minimal realtime API based on websockets (#33187) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-30 18:41:29 +08:00
cmunley1 3bba2edb0f support returning tokenids in responses api (#33212) 
Signed-off-by: Christian Munley <cmunley@nvidia.com>
 2026-01-29 16:52:39 +08:00
Didier Durandandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 31b25f6516 [Doc]: fixing multiple typos in diverse files (#33256) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Didier Durand <2927957+didier-durand@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 16:52:03 +08:00
wangln19 39037d258e Fix tool call indexing double-counting (#33141) 
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
 2026-01-29 05:57:09 +00:00
Harry Mellor 2eb673a088 Add flake8-implicit-str-concat rules to Ruff (#33191) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-28 04:56:10 +00:00
wang.yuqi 7cbbca9aaa [Frontend] Cleanup api server (#33158) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
 2026-01-27 15:18:10 +00:00
wang.yuqiandCyrus Leung 76139d0801 [Frontend] Frontend will only attach supported tasks corresponding entrypoints. (#33139) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-27 12:15:43 +00:00
Cyrus Leung e0b005d9cf [Frontend] Cleanup serving engine (#33103) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 20:47:26 -08:00
Cyrus Leung c831911be2 [Frontend] Reduce mixin usage in serving pooling (#33101) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-27 11:50:37 +08:00
wangln19Roger WangIsotr0py
 		2d7053438a fix: preserve native tool call ID in multi-turn tool calling (#32768) 
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-01-27 10:22:35 +08:00
Jared Wen 6ee7f18f33 [Logging] add --disable-access-log-for-endpoints CLI option (#30011) 
Add a new CLI option --disable-access-log-for-endpoints to suppress
uvicorn access logs for specified endpoints (e.g., /health, /metrics, /ping).

This addresses the need to reduce log noise in production environments
where health check endpoints are frequently polled by load balancers or
monitoring systems, generating excessive log entries that obscure
meaningful request logs.

Fixes #29982

Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-01-26 21:49:03 +00:00
7. Sun 0f19427db5 [Perf] Cache exc.errors() result in validation exception handler (#32984) 
Signed-off-by: 7. Sun <jhao.sun@gmail.com>
 2026-01-24 02:01:35 -08:00
Cyrus Leung 51931c5c9a [UX] Deduplicate sampling parameter startup logs (#32953) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-24 17:37:28 +08:00
Wentao Ye 37c9859fab [Refactor] Clean up unused variables & func (#32692) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-23 17:04:25 -05:00
sangbumlikeagod 9b77bb790d [Frontend] add logprob, compression_rate to 'verbose_json' features (#31059) 
Signed-off-by: sangbumlikeagod <oironese@naver.com>
Signed-off-by: sangbumlikeagod <98077576+sangbumlikeagod@users.noreply.github.com>
 2026-01-23 16:35:13 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 05f3d714db [Frontend][3/n] Make pooling entrypoints request schema consensus | EmbedRequest & ClassifyRequest (#32905) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 12:03:44 +00:00
Nick Hill 7fe255889e [Misc] Log vLLM logo when starting server (#32796) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-23 11:15:12 +08:00
RickyChen / 陳昭儒 69d09fdd6c [Feature] Add --ssl-ciphers CLI argument for TLS cipher control (#30937) 
Signed-off-by: rickychen-infinirc <ricky.chen@infinirc.com>
 2026-01-22 09:53:24 -08:00
Tyler Michael Smith 803e3f3f68 [UX] Default api_server_count to dp_size if not specified (#32525) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-01-22 17:35:35 +00:00
Cyrus Leung d117a4d1a9 [Frontend] Introduce Renderer for processing chat messages (using ModelConfig) (#30200) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 12:44:22 +00:00
Chauncey 841d53aaa8 [Frontend] add prompt_cache_key for openresponses (#32824) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-01-22 11:34:14 +00:00
Jackmin801andJee Jee Li 12dab78f49 [Feat] allow inplace loading lora (#31326) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-20 10:15:20 +08:00
wang.yuqi c88860d759 [Frontend] Score entrypoint support data_1 & data_2 and queries & documents as inputs (#32577) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-19 14:07:46 +00:00
Hyunkyun Moonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 3c8740aacb [Frontend] Add render endpoints for prompt preprocessing (#32473) 
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
Signed-off-by: Hyunkyun Moon <mhg5303@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-19 12:21:46 +08:00