iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1050 Commits
Author SHA1 Message Date
Ronen Schaffer cd5601ac37 [BugFix] Prevent exporting duplicate OpenTelemetry spans (#9017) 2024-10-22 11:11:53 -07:00
Woosuk Kwon 6c5af09b39 [V1] Implement vLLM V1 [1/N] (#9289) 2024-10-22 01:24:07 -07:00
Travis JohnsonandCyrus Leung b729901139 [Bugfix]: serialize config by value for --trust-remote-code (#6751) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-10-21 19:46:24 -07:00
youkaichao 76a5e13270 [core] move parallel sampling out from vllm core (#9302) 2024-10-22 00:31:44 +00:00
Wallas Henrique 711f3a7806 [Frontend] Don't log duplicate error stacktrace for every request in the batch (#9023) 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
 2024-10-21 14:49:41 -07:00
Joe Runde 82c25151ec [Doc] update gpu-memory-utilization flag docs (#9507) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-10-19 11:26:36 +08:00
KunjanandKunjan Patel 9bb10a7d27 [MISC] Add lora requests to metrics (#9477) 
Co-authored-by: Kunjan Patel <kunjanp_google_com@vllm.us-central1-a.c.kunjanp-gke-dev-2.internal>
 2024-10-18 20:50:18 +00:00
Cyrus Leung 051eaf6db3 [Model] Add user-configurable task for models that support both generation and embedding (#9424) 2024-10-18 11:31:58 -07:00
Nick Hill 1ffc8a7362 [BugFix] Typing fixes to RequestOutput.prompt and beam search (#9473) 2024-10-18 07:19:53 +00:00
Kuntai Du 81ede99ca4 [Core] Deprecating block manager v1 and make block manager v2 default (#8704) 
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
 2024-10-17 11:38:15 -05:00
Russell Bryant 776dbd74f1 [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-16 22:55:59 +00:00
Cyrus Leung cee711fdbb [Core] Rename input data types (#8688) 2024-10-16 10:49:37 +00:00
Cyrus Leung 7e7eae338d [Misc] Standardize RoPE handling for Qwen2-VL (#9250) 2024-10-16 13:56:17 +08:00
Brendan Wong 4d31cd424b [Frontend] merge beam search implementations (#9296) 2024-10-14 15:05:52 -07:00
Wallas Henrique 8baf85e4e9 [Doc] Compatibility matrix for mutual exclusive features (#8512) 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
 2024-10-11 11:18:50 -07:00
Tyler Michael Smith 7342a7d7f8 [Model] Support Mamba (#6484) 2024-10-11 15:40:06 +00:00
Sebastian Schoennenbeck df3dcdf49d [Bugfix] Fix priority in multiprocessing engine (#9277) 2024-10-11 15:35:35 +00:00
Cyrus Leung e808156f30 [Misc] Collect model support info in a single process per model (#9233) 2024-10-11 11:08:11 +00:00
youkaichaoandBrendan Wong cbc2ef5529 [misc] hide best_of from engine (#9261) 
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
 2024-10-10 21:30:44 -07:00
Russell Bryant cf25b93bdd [Core] Fix invalid args to _process_request (#9201) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-10 12:10:09 +08:00
Alex Brooks a3691b6b5e [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-08 14:12:56 +00:00
Brendan Wong 8c746226c9 [Frontend] API support for beam search for MQLLMEngine (#9117) 2024-10-08 05:51:43 +00:00
youkaichao 18b296fdb2 [core] remove beam search from the core (#9105) 2024-10-07 05:47:04 +00:00
sroy745 c8f26bb636 [BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103) 2024-10-07 03:52:42 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath cb3b2b9ba4 [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-10-06 12:48:11 -07:00
Brendan Wongandyoukaichao 168cab6bbf [Frontend] API support for beam search (#9087) 
Co-authored-by: youkaichao <youkaichao@126.com>
 2024-10-05 23:39:03 -07:00
Chen ZhangandRoger Wang dac914b0d6 [Bugfix] use blockmanagerv1 for encoder-decoder (#9084) 
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-10-05 04:45:38 +00:00
Roger WangandDarkLight1337 26aa325f4f [Core][VLM] Test registration for OOT multimodal models (#8717) 
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-04 10:38:25 -07:00
Michael Goin 303d44790a [Misc] Enable multi-step output streaming by default (#9047) 2024-10-03 22:55:42 -04:00
xendoandJerzy Zagorski 63e39937f9 [Frontend] [Neuron] Parse literals out of override-neuron-config (#8959) 
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
 2024-10-03 18:02:07 +00:00
sroy745 f5d72b2fc6 [Core] Make BlockSpaceManagerV2 the default BlockManager to use. (#8678) 2024-10-03 09:44:21 -07:00
afeldman-nmVarun Sundar RabindranathAndrew Feldman
 		563649aafe [Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804) 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
 2024-10-02 07:52:20 +00:00
Lily Liu 1570203864 [Spec Decode] (1/2) Remove batch expansion (#8839) 2024-10-01 16:04:42 -07:00
Sebastian Schoennenbeck 35bd215168 [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) 2024-10-01 09:58:06 +00:00
Joe RundeandNick Hill 062c89e7c9 [Frontend][Core] Move guided decoding params into sampling params (#8252) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
 2024-10-01 09:34:25 +08:00
Sebastian Schoennenbeck be76e5aabf [Core] Make scheduling policy settable via EngineArgs (#8956) 2024-09-30 12:28:44 +00:00
Varun Sundar Rabindranath 19d02ff938 [Bugfix] Fix PP for Multi-Step (#8887) 2024-09-28 08:52:46 -07:00
Sebastian Schoennenbeck bd429f2b75 [Core] Priority-based scheduling in async engine (#8850) 2024-09-27 15:07:10 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath c2ec430ab5 [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-09-27 13:32:07 -07:00
Cyrus Leung 3b00b9c26c [Core] renamePromptInputs and inputs (#8876) 2024-09-26 20:35:15 -07:00
Chen Zhangsimon-moChang SuSimon MoRoger WangRoger Wang
 		770ec6024f [Model] Add support for the multi-modal Llama 3.2 model (#8811) 
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-09-25 13:29:32 -07:00
Simon Mo 4f1ba0844b Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) 2024-09-25 10:36:26 -07:00
科英 64840dfae4 [Frontend] MQLLMEngine supports profiling. (#8761) 2024-09-25 09:37:41 -07:00
Cyrus Leung 28e1299e60 rename PromptInputs and inputs with backward compatibility (#8760) 2024-09-25 09:36:47 -07:00
Joe Runde 6e0c9d6bd0 [Bugfix] Use heartbeats instead of health checks (#8583) 2024-09-24 20:37:38 -07:00
Archit Patke 6da1ab6b41 [Core] Adding Priority Scheduling (#5958) 2024-09-24 19:50:50 -07:00
Travis Johnson 01b6f9e1f0 [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-09-24 17:29:56 -07:00
Simon Mo 3185fb0cca Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (#8750) 2024-09-24 05:45:20 +00:00
Alexander Matveev 1a2aef3e59 Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) 2024-09-23 15:38:04 -07:00
Alex Brooks 9b8c8ba119 [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-09-23 07:44:48 +00:00