iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,220 Commits 3 Branches 0 Tags
00f7f258282ad3ea400a760ba3bc4679ef4e40fe
Commit Graph
1083 Commits
Author SHA1 Message Date
Yufeng HeandChauncey e85dd21497 fix: report stop_sequence stop_reason in Anthropic Messages API (#45807) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-20 03:23:43 +00:00
Andreas KaratzasandOpenAI Codex cb58bb9c1e [CI] Harden RemoteVLLMServer GPU cleanup checks (#52282) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 14:17:17 -05:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3a386cfaf5 [ROCm] Give EngineCore cleanup grace after request abort (#52281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-19 11:39:59 -05:00
Quentin Gallouédec 92bdee05cb [Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399) 
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
 2026-08-19 20:48:45 +08:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
LuckyStep 9a9aa2b017 [Bugfix] Redact api_key in non-default args log (#52523) 
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com>
 2026-08-19 15:52:53 +08:00
Vineeth Sai Varikuntla 86a89a9348 [Bugfix][Frontend] Run the serve arg checks for vllm launch too (#52825) 
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
 2026-08-19 04:48:50 +00:00
Yufeng He 5c9ff5366b [Bugfix] Accept logprobs=-1 in the Completion API (#46175) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-08-18 08:22:13 +00:00
Rajath Pai b01728b088 [Bugfix] Return 4xx for client-caused errors in /detokenize (#52622) 
Signed-off-by: rajathpi <rajathpai2000@gmail.com>
 2026-08-18 07:07:15 +00:00
flb_ d785eb51ce [Test] Add pause/resume E2E tests (#52144) 
Signed-off-by: flb_ <floatlibai@gmail.com>
 2026-08-18 14:46:44 +08:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
OlaandOpenAI Codex 95901ce70a fix(pooling): validate BGE-M3 combined task ownership (#51823) 
Signed-off-by: Zhe Li <2843409461@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-17 10:30:35 +00:00
wang.yuqi 93550cc4cd [Frontend] Consolidate entrypoint middleware (#52309) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-17 06:17:15 +00:00
Kaif Kohari a6a2a93f9b [Bugfix][Frontend] Guard remaining before-validators against non-object JSON bodies (#52528) 
Signed-off-by: Kaif <kaifkohari10@gmail.com>
 2026-08-17 02:27:59 +00:00
Do_it_now_! 70aaec832b [Bugfix][Anthropic] Return 4xx for client-caused errors in /v1/messages (#52246) 
Signed-off-by: Do_it_now_! <lizhengcheng233@gmail.com>
 2026-08-16 01:05:17 -07:00
Jeffrey WangandClaude Opus 5 41f12a0daf [Bugfix] Raise VLLMValidationError from structured output validators (#52394) 
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-16 01:04:51 -07:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-14 09:27:24 +00:00
Fangchen LiandClaude Opus 5 103c419f04 [Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277) 
Signed-off-by: Fangchen Li <fangchen.li@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 09:15:59 +00:00
Junhao ShenClaude Opus 4.8Vadim Gimpelson
 		b216db3ed0 [Bugfix] Reject negative token ids as out-of-vocabulary (#51795) 
Signed-off-by: Junhao Shen <junshen@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-08-14 01:07:29 +00:00
Andreas Karatzas b96bcd0b47 [ROCm][CI] Solidify entrypoint LLM lifecycle (#51280) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-13 12:24:02 -05:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) 
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
 2026-08-13 13:11:51 +00:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Frank 7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-13 06:13:25 +00:00
Kevin H. LuuandOpenAI Codex 8eb35c5217 [CI] Mirror external test assets in vLLM S3 (#52064) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-13 02:08:59 +00:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
Aaron HaoandClaude Opus 5 793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 18:53:15 -07:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
cherry77-cloudandOpenAI Codex 65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 04:49:49 +00:00
Frank adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-11 04:12:22 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Tarun Kumar 48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-08-11 01:31:19 +00:00
Taneem IbrahimandWentao Ye ea3115e30b [CI] Stabilize DP supervisor lifecycle tests (#51557) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-10 09:13:52 -04:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
ChaunceyandNick Hill 72c0d67657 [Feat] Support thinking_token_budget in Model Runner V2 (#46727) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 03:36:18 +00:00
Nick Hill 77b519912b [CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock (#51050) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-04 20:29:35 -05:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Omkar ShewaleAndreas KaratzasAndreas Karatzas
 		f5bb701fa2 [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-03 13:33:14 +08:00
yzong-rh dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-01 12:10:26 -04:00
Cao Qian 81a42d37b1 [Frontend] Add cache_salt support to Anthropic Messages API (#49498) 
Signed-off-by: aeon-x <talexcao@gmail.com>
 2026-08-01 07:36:42 +00:00
andrewbcohereandCyrus Leung 9c110fa522 [Frontend] Cohere chat v2 api support (#47189) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-31 22:53:54 -07:00
yzong-rhandBen Browning e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-31 15:45:40 -04:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Nicolò Lucchesi 8700f86a70 [CI] Fix tests/entrypoints/multimodal/openai/chat_completion/test_audio.py::test_chat_streaming_audio (#50451) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-30 16:18:53 -07:00
Andreas Karatzas 5b95890742 [FlexAttention] Avoid encoder block-mask compile explosion (#50339) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 11:24:13 -04:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Andreas Karatzas e04a30a77c [Frontend] Lazily initialize chat media connectors (#49914) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 14:56:37 +08:00
Martin Hickey 0a31372e5f [Frontend] Add detokenization streaming derender for disaggregated serving (#47301) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-29 18:01:20 -07:00