iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,987 Commits 3 Branches 0 Tags
925ea7e60f727bbb3fe3fcdb1c02b01a5bfb2749
Commit Graph
1760 Commits
Author SHA1 Message Date
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-14 09:27:24 +00:00
Fangchen LiandClaude Opus 5 103c419f04 [Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277) 
Signed-off-by: Fangchen Li <fangchen.li@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 09:15:59 +00:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) 
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
 2026-08-13 13:11:51 +00:00
Taneem Ibrahim d0ae25e2f2 [Bugfix] Preserve Anthropic disable_parallel_tool_use (#52021) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-13 08:25:55 -04:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Frank 7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-13 06:13:25 +00:00
Taneem Ibrahim 6ea6c42659 [Bugfix] Bound Anthropic stop sequences (#51997) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-12 12:05:12 -04:00
Russell BryantandClaude Opus 4.8 8151f2ad43 [Docs] Warn that --api-key does not gate all endpoints (#51999) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-12 08:16:35 -07:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
Clinton ThomasLucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b64a2708b0 Bound generation inputs before expensive work (#51447) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 22:03:50 +08:00
cherry77-cloudandOpenAI Codex 65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 04:49:49 +00:00
Frank adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-11 04:12:22 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Tarun Kumar 48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-08-11 01:31:19 +00:00
Myeongwon Kim 8977ea8895 [Perf] Skip detokenization in offline beam search (#50333) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
 2026-08-10 16:17:38 -07:00
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fa722b9f01 [Rust Frontend][gRPC] Add explicit data-parallel rank routing (#51178) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-10 15:05:24 -07:00
Juan Pérez de AlgabaandCursor 640a09086d Fix DoS via sample-rate forgery bypassing audio decode duration guard (#49948) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-10 15:49:17 +00:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
Bugen ZhaoOpenAI CodexNick Hill
 		7b9f2dad89 [Frontend] Watch frontend processes during engine startup (#43417) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-06 11:17:24 -07:00
Chauncey 22170354d8 [Feature] Parse request priority from HTTP header (#51089) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-05 21:10:29 -07:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Wentao Ye 0cf49a5d15 [Refactor] Remove multiple dead codes (#50285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-08-03 10:59:08 -04:00
Cyrus Leung b3f97dae24 [Frontend] Require cache_salt to be non-empty via schema (#50816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-03 06:51:27 +00:00
Omkar ShewaleAndreas KaratzasAndreas Karatzas
 		f5bb701fa2 [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-03 13:33:14 +08:00
yzong-rh dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-01 12:10:26 -04:00
Cao Qian 81a42d37b1 [Frontend] Add cache_salt support to Anthropic Messages API (#49498) 
Signed-off-by: aeon-x <talexcao@gmail.com>
 2026-08-01 07:36:42 +00:00
andrewbcohereandCyrus Leung 9c110fa522 [Frontend] Cohere chat v2 api support (#47189) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-31 22:53:54 -07:00
Ning Xie 0c4dc7c488 [graceful shutdown] fix http server start firstly before app signal handler register (#49668) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 20:51:33 +00:00
yzong-rhandBen Browning e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-31 15:45:40 -04:00
Ning Xie 726ef437a1 [chore] delete useless code (#49424) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-07-31 11:54:08 -07:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
wang.yuqi f5ffc59b6a [Renderer] Warm up the renderer properly. (#50408) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-31 11:46:53 +00:00
Nicolò Lucchesi 17beffd55f [Misc] Clarify mono audio requirement (#50141) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-31 10:46:53 +00:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Andreas Karatzas e04a30a77c [Frontend] Lazily initialize chat media connectors (#49914) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 14:56:37 +08:00
Martin Hickey 0a31372e5f [Frontend] Add detokenization streaming derender for disaggregated serving (#47301) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-29 18:01:20 -07:00
christian 9d1aa4dda8 [Frontend] Add diarized_json support for MOSS-Transcribe-Diarize (#48543) 
Signed-off-by: Lucas Christian <lucaschgf7@gmail.com>
 2026-07-29 23:24:30 +00:00
christian e0cfa52d22 [Bugfix][Frontend] Return transcription and translation verbose as float (#49073) 
Signed-off-by: Lucas Christian <lucaschgf7@gmail.com>
 2026-07-29 13:16:13 +00:00
Bugen Zhao f5a7cce9b6 [Model] Add Kimi K3 support: Python frontend [2/2] (#50093) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-29 01:06:01 -07:00
Zach Zhu 58f9659397 [Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-07-29 04:20:40 +00:00
Bugen Zhao 01661cc57f [Rust][Benchmark] Make vllm bench serve Rust delegation opt-in (#50081) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 16:21:12 +00:00
Itay Alroy 35efdf6b34 [Elastic EP] Async preparation (#47288) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-07-28 05:19:13 -07:00
Shuolei Wang 9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
 2026-07-28 14:35:29 +08:00
TobyJBell 15d65f8669 [Bugfix] Changed speech to text chunk timestamp to cumulative approach (#41131) 
Signed-off-by: Toby Bell <toby.bell1702@hotmail.co.uk>
 2026-07-27 17:06:46 +00:00
Umut Polat 27d7061ef6 [Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online (#49963) 
Signed-off-by: Umut Polat <52835619+umut-polat@users.noreply.github.com>
 2026-07-27 22:52:59 +08:00
Walter Beller-Moralesandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8040ef2426 [Frontend] expose stream_interval as req sampling param (#49754) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-27 11:30:37 +08:00
fangyuchuzWaNg3a798347923TianZhuo205150940a798347923w00689259zWaNg3yzchang-plusJade Zheng
 		0b0bd2b5f6 [Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: a798347923 <2645302020@qq.com>
Signed-off-by: TianZhuo <2770730562@qq.com>
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com>
Signed-off-by: w00689259 <wangzhuo66@huawei.com>
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Signed-off-by: yzchang-plus <1078477584@qq.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Co-authored-by: a798347923 <2645302020@qq.com>
Co-authored-by: TianZhuo <2770730562@qq.com>
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com>
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Co-authored-by: w00689259 <wangzhuo66@huawei.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: yzchang-plus <1078477584@qq.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
 2026-07-25 11:49:09 -04:00
Harry Mellor 3e74c60b9c [Docs] Use gen-files for generated docs content (#49587) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-25 14:43:21 +00:00
Guan-Ming Chiu 80c7683923 [Perf] Defer MM embeds loading off the event loop (#49477) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-07-23 13:58:50 +00:00