iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,987 Commits 3 Branches 0 Tags
925ea7e60f727bbb3fe3fcdb1c02b01a5bfb2749
Commit Graph
1066 Commits
Author SHA1 Message Date
wang.yuqi b8165e5e58 [Frontend] Consolidate entrypoint exception handler (#52261) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-14 09:27:24 +00:00
Fangchen LiandClaude Opus 5 103c419f04 [Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277) 
Signed-off-by: Fangchen Li <fangchen.li@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 09:15:59 +00:00
Junhao ShenClaude Opus 4.8Vadim Gimpelson
 		b216db3ed0 [Bugfix] Reject negative token ids as out-of-vocabulary (#51795) 
Signed-off-by: Junhao Shen <junshen@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-08-14 01:07:29 +00:00
Andreas Karatzas b96bcd0b47 [ROCm][CI] Solidify entrypoint LLM lifecycle (#51280) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-13 12:24:02 -05:00
aoshen02OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7553aac77b [Frontend] Add routed-experts prompt offset (#51906) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-13 08:09:00 -07:00
yang rui 152c913a63 [Frontend] Log output token IDs at DEBUG level (#52098) 
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com>
 2026-08-13 13:11:51 +00:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
Frank 7bc1660bd6 [Misc] Use VLLMValidationError in pooling input validation (#51931) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-13 06:13:25 +00:00
Kevin H. LuuandOpenAI Codex 8eb35c5217 [CI] Mirror external test assets in vLLM S3 (#52064) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-13 02:08:59 +00:00
Vinay R DamodaranandClaude Opus 5 3ee2df3033 [Frontend] Make model optional on all /derender request classes (#51463) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 04:48:58 +00:00
Aaron HaoandClaude Opus 5 793ca6998a [Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729) 
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-11 18:53:15 -07:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
cherry77-cloudandOpenAI Codex 65b7662d3f [Bugfix][Frontend] Report Cohere stop sequences correctly (#51556) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 04:49:49 +00:00
Frank adc1200597 [Misc] Use VLLMValidationError in scoring input validation (#51753) 
Signed-off-by: frank-suwen <suwenw2@outlook.com>
 2026-08-11 04:12:22 +00:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Tarun Kumar 48bada6ea4 Fix chat completion 500 on non-object JSON bodies (#51654) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-08-11 01:31:19 +00:00
Taneem IbrahimandWentao Ye ea3115e30b [CI] Stabilize DP supervisor lifecycle tests (#51557) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-08-10 09:13:52 -04:00
Nick Hill ac70ce96e0 [Frontend] Disable uvicorn signal handlers instead of racing them (#50916) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 13:40:24 -06:00
ChaunceyandNick Hill 72c0d67657 [Feat] Support thinking_token_budget in Model Runner V2 (#46727) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-07 03:36:18 +00:00
Nick Hill 77b519912b [CI][Bugfix] Fix test_shutdown_on_engine_failure startup deadlock (#51050) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-04 20:29:35 -05:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
Omkar ShewaleAndreas KaratzasAndreas Karatzas
 		f5bb701fa2 [Bugfix][Frontend] Constrain Anthropic cache_salt to non-empty (#50764) 
Signed-off-by: Omkar Shewale <omkarshewale2001@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-03 13:33:14 +08:00
yzong-rh dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-01 12:10:26 -04:00
Cao Qian 81a42d37b1 [Frontend] Add cache_salt support to Anthropic Messages API (#49498) 
Signed-off-by: aeon-x <talexcao@gmail.com>
 2026-08-01 07:36:42 +00:00
andrewbcohereandCyrus Leung 9c110fa522 [Frontend] Cohere chat v2 api support (#47189) 
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-31 22:53:54 -07:00
yzong-rhandBen Browning e8b358b4c0 [Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-31 15:45:40 -04:00
Recoordinateandlatent-9 0b5b49dde8 [Bugfix][Frontend] Raise VLLMValidationError for user-facing errors in chat_utils.py (#50491) 
Signed-off-by: latent-9 <296084221+latent-9@users.noreply.github.com>
Co-authored-by: latent-9 <296084221+latent-9@users.noreply.github.com>
 2026-07-31 11:54:43 +00:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Nicolò Lucchesi 8700f86a70 [CI] Fix tests/entrypoints/multimodal/openai/chat_completion/test_audio.py::test_chat_streaming_audio (#50451) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-30 16:18:53 -07:00
Andreas Karatzas 5b95890742 [FlexAttention] Avoid encoder block-mask compile explosion (#50339) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 11:24:13 -04:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Andreas Karatzas e04a30a77c [Frontend] Lazily initialize chat media connectors (#49914) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-30 14:56:37 +08:00
Martin Hickey 0a31372e5f [Frontend] Add detokenization streaming derender for disaggregated serving (#47301) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-29 18:01:20 -07:00
Seiji Eicher 6370e53f24 [Frontend] Reuse prefill token ids on the decode chat path for disaggregated serving (#48145) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2026-07-29 10:02:57 +02:00
Zach Zhu 58f9659397 [Frontend][Core] Standardize request error handling with VLLMError hierarchy (#49665) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-07-29 04:20:40 +00:00
Andreas Karatzas 176256b962 [ROCm][CI] Stabilize ROCm audio streaming test (#50163) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-29 07:58:10 +08:00
Bugen Zhao 01661cc57f [Rust][Benchmark] Make vllm bench serve Rust delegation opt-in (#50081) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 16:21:12 +00:00
Shuolei Wang 9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
 2026-07-28 14:35:29 +08:00
TobyJBell 15d65f8669 [Bugfix] Changed speech to text chunk timestamp to cumulative approach (#41131) 
Signed-off-by: Toby Bell <toby.bell1702@hotmail.co.uk>
 2026-07-27 17:06:46 +00:00
Umut Polat 27d7061ef6 [Bugfix] Restore truncate_prompt_tokens for Jina rerank/score online (#49963) 
Signed-off-by: Umut Polat <52835619+umut-polat@users.noreply.github.com>
 2026-07-27 22:52:59 +08:00
Taneem Ibrahim 0164022c90 [CI] Fix speech correctness check rejecting improved WER (#49853) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-07-26 06:24:07 +00:00
Chang Guo 7a6a5b3667 [CI] Compute speech WER directly with jiwer (#49773) 2026-07-25 20:51:21 -04:00
Taneem Ibrahim 6b0103d1c9 [CI] Stabilize Pooling Rerank Equivalence Test (#49822) 2026-07-25 14:36:50 -04:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
djramicAndreas KaratzasOpenAI Codex
 		7513d071bd [ROCm][CI] Fix XPASS(strict) on mixed audio embeds test (#49733) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-24 16:09:18 -05:00
Harry Mellor 89f6aa3a9e [KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flaky api-server test (#49734) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-24 13:56:57 -07:00
Jiangyun ZhuandOpenAI Codex 9e6746b3c7 [CI] Stabilize memory-sensitive compile and structured output tests (#49749) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-24 16:45:09 -04:00
Andreas Karatzas a49d37c6b9 [CI] Disable reasoning in Responses smoke test (#49511) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-23 19:02:54 +00:00
Andrey TalmanAndrey TalmanClaude
 		75ccdf3145 [Core] Update PyTorch to 2.13.0, torchvision to 0.28.0, triton to 3.7.1 (#48155) 
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-23 11:09:36 -07:00
wang.yuqi 61c9ef986a [Frontend] Parallelize preprocessing within the same request for pooling models online serving. (#49153) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-07-22 10:56:23 +00:00