iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
20,383 Commits 3 Branches 0 Tags
afc91fa0d2eb8bc050d80e9bcd74c8b1d72b4191
Commit Graph
1099 Commits
Author SHA1 Message Date
wang.yuqi 735daf8ef6 [Frontend] Move cli_args.py and dp_supervisor.py out openai folder (#53659) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-25 07:02:32 +00:00
Moe Huzaifa d3e2888c75 [Bugfix][Frontend] Redact hf_token in the non-default args log (#53625) 
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com>
 2026-08-25 03:06:46 +00:00
rasmith a0f1b9ad05 [CI][The Rock] Increase flex attention abs tol (#53646) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-08-24 23:57:03 +00:00
wang.yuqi 9cef631f30 [Frontend] Move run_batch.py out openai folder (#53500) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-24 15:04:10 +00:00
andrewbcohere 41ae917c12 Add Cohere ChatV2 render endpoint (#53219) 2026-08-24 12:27:57 +00:00
Tan ChaoandClaude e239947777 [Bugfix][Frontend] Fix run_batch upload retrying on success and unawaited error body (#50588) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-24 11:33:44 +00:00
Clinton ThomasandLucas Bourtoule 2ec6f0d71e Reject oversized media before fully downloading it (#51896) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
 2026-08-24 01:15:09 +00:00
Dieter Schwarzmann 585bb07c70 feat: add SSE keep-alive comments for idle streaming responses (#51034) 
Signed-off-by: fuzzifikation <40043768+fuzzifikation@users.noreply.github.com>
 2026-08-23 18:13:02 -07:00
FrankandCodex cd329413e2 [Misc] Use VLLMValidationError in Cohere request validation (#52467) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-08-24 00:52:27 +00:00
Jensen ChenandClaude Opus 5 39eba6ac36 [Misc] Use VLLMValidationError in offline inference input validation (#53432) 
Signed-off-by: Jensen Chen <a1043904820@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 08:43:51 -07:00
Maroon Ayoub a556f3fccb Forward Anthropic vllm_xargs to sampling params (#53308) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
 2026-08-21 15:23:21 -07:00
andrewbcohere f94dcde5c5 Fix Cohere ChatV2 citation and tool handling issues (#52175) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
 2026-08-21 15:23:17 -07:00
Tan Chao 1baf372bfc [Frontend] Use VLLMValidationError for batch request URL validation (#50191) 
Signed-off-by: Chao Tan <chaos.tc@gmail.com>
 2026-08-21 09:07:24 -07:00
Wei-Cheng (Wayne) Chiu b41dc4ec72 [Bugfix] Return HTTP 500 for non-streaming generate errors (#49195) 
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
 2026-08-21 09:07:17 -07:00
Greg Pereira 18aa245a99 using existing uvicorn configuration for dp supervisor (#52473) 
Signed-off-by: Gregory Pereira <grpereir@redhat.com>
 2026-08-21 09:06:54 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 14:17:17 -05:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) 
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
 2026-08-19 15:52:53 +08:00
Vineeth Sai Varikuntla 86a89a9348 [Bugfix][Frontend] Run the serve arg checks for vllm launch too (#52825) 
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
 2026-08-19 04:48:50 +00:00
Yufeng He 5c9ff5366b [Bugfix] Accept logprobs=-1 in the Completion API (#46175) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-08-18 08:22:13 +00:00
Rajath Pai b01728b088 [Bugfix] Return 4xx for client-caused errors in /detokenize (#52622) 
Signed-off-by: rajathpi <rajathpai2000@gmail.com>
 2026-08-18 07:07:15 +00:00
flb_ d785eb51ce [Test] Add pause/resume E2E tests (#52144) 
Signed-off-by: flb_ <floatlibai@gmail.com>
 2026-08-18 14:46:44 +08:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
OlaandOpenAI Codex 95901ce70a fix(pooling): validate BGE-M3 combined task ownership (#51823) 
Signed-off-by: Zhe Li <2843409461@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-17 10:30:35 +00:00
wang.yuqi 93550cc4cd [Frontend] Consolidate entrypoint middleware (#52309) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-17 06:17:15 +00:00
Kaif Kohari a6a2a93f9b [Bugfix][Frontend] Guard remaining before-validators against non-object JSON bodies (#52528) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
 2026-08-17 02:27:59 +00:00
Do_it_now_! 70aaec832b [Bugfix][Anthropic] Return 4xx for client-caused errors in /v1/messages (#52246) 
Signed-off-by: Do_it_now_! <lizhengcheng233@gmail.com>
 2026-08-16 01:05:17 -07:00
Jeffrey WangandClaude Opus 5 41f12a0daf [Bugfix] Raise VLLMValidationError from structured output validators (#52394) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-16 01:04:51 -07:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-14 09:27:24 +00:00
Fangchen LiandClaude Opus 5 103c419f04 [Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277) 
Signed-off-by: Fangchen Li <fangchen.li@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 09:15:59 +00:00
Junhao ShenClaude Opus 4.8Vadim Gimpelson
 		b216db3ed0 [Bugfix] Reject negative token ids as out-of-vocabulary (#51795) 
Signed-off-by: Junhao Shen <junshen@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-08-14 01:07:29 +00:00
Andreas Karatzas b96bcd0b47 [ROCm][CI] Solidify entrypoint LLM lifecycle (#51280) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-13 12:24:02 -05:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) 
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
 2026-08-13 13:11:51 +00:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Frank 7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-13 06:13:25 +00:00
Kevin H. LuuandOpenAI Codex 8eb35c5217 [CI] Mirror external test assets in vLLM S3 (#52064) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-13 02:08:59 +00:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
Aaron HaoandClaude Opus 5 793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 18:53:15 -07:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
cherry77-cloudandOpenAI Codex 65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 04:49:49 +00:00
Frank adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-11 04:12:22 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Tarun Kumar 48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-08-11 01:31:19 +00:00
Taneem IbrahimandWentao Ye ea3115e30b [CI] Stabilize DP supervisor lifecycle tests (#51557) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-10 09:13:52 -04:00