iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
1816 Commits
Author SHA1 Message Date
Clinton ThomasandLucas Bourtoule ee17d0d869 Use server-generated keys for late-interaction query caches (#51445) 
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-09-03 07:15:58 +00:00
Chauncey b2558f8da3 [Bugfix] Fix launch render hanging on shutdown (#54913) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-02 08:15:35 +00:00
Francisco Javier Arceo c00091e026 [Frontend] Gate scale-out endpoints behind opt-in flag (#54579) 
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
 2026-09-02 14:31:14 +08:00
Chauncey 339e16cbb6 [Bugfix] Support MCP SDK 2.x tool input schemas (#53870) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-01 11:02:39 -04:00
lzhan011 fa99a6fea6 [Bugfix][Security] Bound the validation-error response body (#54684) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
 2026-09-01 06:51:16 +00:00
Nicolò Lucchesi dafbef15a1 [Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-31 14:26:49 +00:00
waizuichougou 810bc3250c [Frontend][Performance] Resolve async media across modalities concurrently (#54537) 
Signed-off-by: waizuichougou <2082431897@qq.com>
 2026-08-31 10:50:17 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
eligottsandOpenAI Codex 4ae172231c [Frontend] Forward cache salt for content parts (#54315) 
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-30 23:29:53 -07:00
Hotragn Pettugani 687db59744 [Bugfix][Frontend] Truncate pooling prompts before padding them (#54364) 
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
 2026-08-30 22:12:03 -07:00
Isotr0py 555ea65e8c [Frontend] Add video embeds input support (#54242) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-31 03:20:47 +00:00
Taneem Ibrahim fe6db3ed5e [Bugfix] Validate scale-out transfer params (#54324) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-30 09:46:12 -04:00
Juan Pérez de Algaba 1dc464d426 [Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion (#54353) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-08-30 04:28:16 +00:00
Kaif KohariandCyrus Leung 7fbfca2670 [Bugfix][Frontend] Only echo the assistant turn in batched chat completions (#52529) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-29 09:44:17 -07:00
Taneem Ibrahim d7b6a35967 [Bugfix] Handle malformed namespace tools (#53763) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-27 14:18:16 -04:00
Hert4 4a6a3272e8 [Bugfix][Frontend] Let pooling requests set padding (#51157) 
Signed-off-by: Hert4 <ductransa01@gmail.com>
 2026-08-27 11:06:20 +00:00
Ting SUN 50708f95ae [Bugfix][OpenAI] Fix streamed completion logprob offsets with echo (#47815) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-08-27 18:19:07 +08:00
Hung Hoang (黄兴） acd0af90e8 [Bugfix] Raise clear error on interleaved multimodal placeholder overcount (#53999) 
Signed-off-by: hungh <hungh@nvidia.com>
 2026-08-27 09:49:40 +00:00
Venkat Balaji SandBen Browning 161ffd37d2 [Bugfix] Guard tool call argument JSON parsing in chat message postprocessing (#48922) 
Signed-off-by: VBS2004 <venkatbalaji2004@gmail.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-08-26 20:26:00 +00:00
Konstantin DunasandOpenAI Codex c71f6f8a81 [Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644) 
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-26 08:42:52 -07:00
Hyunkyun Moon f25c2692fe [Frontend] Add /v1/messages/render endpoint for the Anthropic Messages API (#45803) 
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
 2026-08-26 11:33:59 +08:00
Moe Huzaifa 217a0b5cef [Bugfix][Frontend] Apply the stop string limit to Cohere requests (#53750) 
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
 2026-08-25 16:41:08 +00:00
Moe Huzaifa 3e833933dc [Bugfix][Frontend] Keep credentials out of the Rust frontend launch log (#53738) 
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
 2026-08-25 16:32:12 +00:00
Ning Xie 23310bc3de [sleep functionality] code refactor about sleep/wake_up (#50431) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-08-25 07:45:20 +00:00
wang.yuqi 735daf8ef6 [Frontend] Move cli_args.py and dp_supervisor.py out openai folder (#53659) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-25 07:02:32 +00:00
Moe Huzaifa d3e2888c75 [Bugfix][Frontend] Redact hf_token in the non-default args log (#53625) 
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
 2026-08-25 03:06:46 +00:00
Jee Jee LiandCyrus Leung 4f686e182a [MISC] Cleanup deprecated parameters (#53559) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 10:19:04 -07:00
wang.yuqi 9cef631f30 [Frontend] Move run_batch.py out openai folder (#53500) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-24 15:04:10 +00:00
andrewbcohere 41ae917c12 Add Cohere ChatV2 render endpoint (#53219) 2026-08-24 12:27:57 +00:00
Tan ChaoandClaude e239947777 [Bugfix][Frontend] Fix run_batch upload retrying on success and unawaited error body (#50588) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-24 11:33:44 +00:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-24 01:15:09 +00:00
Dieter Schwarzmann 585bb07c70 feat: add SSE keep-alive comments for idle streaming responses (#51034) 
Signed-off-by: fuzzifikation <40043768+fuzzifikation@users.noreply.github.com>
 2026-08-23 18:13:02 -07:00
FrankandCodex cd329413e2 [Misc] Use VLLMValidationError in Cohere request validation (#52467) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-08-24 00:52:27 +00:00
Jensen ChenandClaude Opus 5 39eba6ac36 [Misc] Use VLLMValidationError in offline inference input validation (#53432) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 08:43:51 -07:00
Maroon Ayoub a556f3fccb Forward Anthropic vllm_xargs to sampling params (#53308) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-08-21 15:23:21 -07:00
andrewbcohere f94dcde5c5 Fix Cohere ChatV2 citation and tool handling issues (#52175) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
 2026-08-21 15:23:17 -07:00
Tan Chao 1baf372bfc [Frontend] Use VLLMValidationError for batch request URL validation (#50191) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
 2026-08-21 09:07:24 -07:00
Wei-Cheng (Wayne) Chiu b41dc4ec72 [Bugfix] Return HTTP 500 for non-streaming generate errors (#49195) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
 2026-08-21 09:07:17 -07:00
Greg Pereira 18aa245a99 using existing uvicorn configuration for dp supervisor (#52473) 
Signed-off-by: Gregory Pereira <grpereir@redhat.com>
 2026-08-21 09:06:54 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) 
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
 2026-08-19 15:52:53 +08:00
Taneem Ibrahim 842dd8fd96 [Pooling] Use semantic task validation errors (#52867) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-19 07:21:08 +00:00
Vineeth Sai Varikuntla 86a89a9348 [Bugfix][Frontend] Run the serve arg checks for vllm launch too (#52825) 
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
 2026-08-19 04:48:50 +00:00
Itay EtelisandItay Etelis d36cc4254e [Bugfix][Elastic EP] Reject scale below the minimum data parallel size (#52702) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
 2026-08-19 01:37:29 +00:00
Yufeng He 5c9ff5366b [Bugfix] Accept logprobs=-1 in the Completion API (#46175) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-08-18 08:22:13 +00:00
Rajath Pai b01728b088 [Bugfix] Return 4xx for client-caused errors in /detokenize (#52622) 
Signed-off-by: rajathpi <rajathpai2000@gmail.com>
 2026-08-18 07:07:15 +00:00