iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,269 Commits 3 Branches 0 Tags
e00a034ad1d7c8ccce28f135fa9380dd9110e525
Commit Graph
1779 Commits
Author SHA1 Message Date
Wei-Cheng (Wayne) Chiu b41dc4ec72 [Bugfix] Return HTTP 500 for non-streaming generate errors (#49195) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
 2026-08-21 09:07:17 -07:00
Greg Pereira 18aa245a99 using existing uvicorn configuration for dp supervisor (#52473) 
Signed-off-by: Gregory Pereira <grpereir@redhat.com>
 2026-08-21 09:06:54 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) 
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
 2026-08-19 15:52:53 +08:00
Taneem Ibrahim 842dd8fd96 [Pooling] Use semantic task validation errors (#52867) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-19 07:21:08 +00:00
Vineeth Sai Varikuntla 86a89a9348 [Bugfix][Frontend] Run the serve arg checks for vllm launch too (#52825) 
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
 2026-08-19 04:48:50 +00:00
Itay EtelisandItay Etelis d36cc4254e [Bugfix][Elastic EP] Reject scale below the minimum data parallel size (#52702) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
 2026-08-19 01:37:29 +00:00
Yufeng He 5c9ff5366b [Bugfix] Accept logprobs=-1 in the Completion API (#46175) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-08-18 08:22:13 +00:00
Rajath Pai b01728b088 [Bugfix] Return 4xx for client-caused errors in /detokenize (#52622) 
Signed-off-by: rajathpi <rajathpai2000@gmail.com>
 2026-08-18 07:07:15 +00:00
OlaandOpenAI Codex 95901ce70a fix(pooling): validate BGE-M3 combined task ownership (#51823) 
Signed-off-by: Zhe Li <2843409461@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-17 10:30:35 +00:00
wang.yuqi 93550cc4cd [Frontend] Consolidate entrypoint middleware (#52309) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-17 06:17:15 +00:00
Kaif Kohari a6a2a93f9b [Bugfix][Frontend] Guard remaining before-validators against non-object JSON bodies (#52528) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
 2026-08-17 02:27:59 +00:00
Do_it_now_! 70aaec832b [Bugfix][Anthropic] Return 4xx for client-caused errors in /v1/messages (#52246) 
Signed-off-by: Do_it_now_! <lizhengcheng233@gmail.com>
 2026-08-16 01:05:17 -07:00
Jeffrey WangandClaude Opus 5 41f12a0daf [Bugfix] Raise VLLMValidationError from structured output validators (#52394) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-16 01:04:51 -07:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-14 09:27:24 +00:00
Fangchen LiandClaude Opus 5 103c419f04 [Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277) 
Signed-off-by: Fangchen Li <fangchen.li@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 09:15:59 +00:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) 
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
 2026-08-13 13:11:51 +00:00
Taneem Ibrahim d0ae25e2f2 [Bugfix] Preserve Anthropic disable_parallel_tool_use (#52021) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-13 08:25:55 -04:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Frank 7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-13 06:13:25 +00:00
Taneem Ibrahim 6ea6c42659 [Bugfix] Bound Anthropic stop sequences (#51997) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-12 12:05:12 -04:00
Russell BryantandClaude Opus 4.8 8151f2ad43 [Docs] Warn that --api-key does not gate all endpoints (#51999) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 08:16:35 -07:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
Clinton ThomasLucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b64a2708b0 Bound generation inputs before expensive work (#51447) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 22:03:50 +08:00
cherry77-cloudandOpenAI Codex 65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 04:49:49 +00:00
Frank adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-11 04:12:22 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Tarun Kumar 48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-08-11 01:31:19 +00:00
Myeongwon Kim 8977ea8895 [Perf] Skip detokenization in offline beam search (#50333) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
 2026-08-10 16:17:38 -07:00
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fa722b9f01 [Rust Frontend][gRPC] Add explicit data-parallel rank routing (#51178) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-10 15:05:24 -07:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-10 15:49:17 +00:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
Bugen ZhaoOpenAI CodexNick Hill
 		7b9f2dad89 [Frontend] Watch frontend processes during engine startup (#43417) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 11:17:24 -07:00
Chauncey 22170354d8 [Feature] Parse request priority from HTTP header (#51089) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-05 21:10:29 -07:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Wentao Ye 0cf49a5d15 [Refactor] Remove multiple dead codes (#50285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-03 10:59:08 -04:00
Cyrus Leung b3f97dae24 [Frontend] Require cache_salt to be non-empty via schema (#50816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-03 06:51:27 +00:00
Omkar ShewaleAndreas KaratzasAndreas Karatzas
 		f5bb701fa2 [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-03 13:33:14 +08:00
yzong-rh dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-01 12:10:26 -04:00
Cao Qian 81a42d37b1 [Frontend] Add cache_salt support to Anthropic Messages API (#49498) 
Signed-off-by: aeon-x <talexcao@gmail.com>
 2026-08-01 07:36:42 +00:00
andrewbcohereandCyrus Leung 9c110fa522 [Frontend] Cohere chat v2 api support (#47189) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-31 22:53:54 -07:00
Ning Xie 0c4dc7c488 [graceful shutdown] fix http server start firstly before app signal handler register (#49668) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 20:51:33 +00:00
yzong-rhandBen Browning e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-31 15:45:40 -04:00
Ning Xie 726ef437a1 [chore] delete useless code (#49424) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 11:54:08 -07:00