iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
43 Commits
Author SHA1 Message Date
Nick Hill 93ecb8139c [BugFix] Increase TP execute_model timeout (#18558) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-23 10:22:11 +08:00
Rabi Mishra acb54ca8e1 Intialize io_thread_pool attribute in the beginning. (#18331) 
Signed-off-by: rabi <ramishra@redhat.com>
 2025-05-21 20:21:14 -07:00
Nick Hill ed5272cf21 [BugFix] Avoid secondary missing MultiprocExecutor.workers error (#17811) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-07 21:55:04 +00:00
Li, Jiang a6fed02068 [V1][PP] Support PP for MultiprocExecutor (#14219) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: jiang.li <jiang1.li@intel.com>
 2025-05-06 07:58:05 -07:00
Nick Hill 5175b884f7 [BugFix] Remove default multiproc executor collective_rpc timeout (#17000) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-22 23:27:14 +00:00
Robert Shawrshaw@neuralmagic.com <rshaw@neuralmagic.com>Cyrus LeungRussell BryantAndrew Feldmanafeldman-nmNick Hill
 		2b05b8ce69 [V1][Frontend] Improve Shutdown And Logs (#11737) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-16 19:48:34 -07:00
yihong 04149cce27 [BugFix] fix some typos found by typos. (#16314) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-09 03:43:59 -07:00
youkaichao a865bc1ca6 [core] do not send error across process (#16174) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-04-07 19:09:03 -07:00
Nick Hill 15dac210f0 [V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Chen Zhang 93a00d7dde [v1] Refactor KVCacheConfig (#14079) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-03-21 04:56:27 -07:00
Mickaël Seznec a597a57595 [Attention] Flash Attention 3 - fp8 (#14570) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-03-20 01:14:20 -04:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Li, Jiang 02296f420d [Bugfix][V1][Minor] Fix shutting_down flag checking in V1 MultiprocExecutor (#14053) 2025-02-28 22:31:01 -08:00
youkaichao eb24dc4a45 [v1] torchrun compatibility (#13642) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-23 22:47:24 +08:00
youkaichao 3e472d882a [core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Woosuk Kwon 4fb8142a0e [V1][PP] Enable true PP with Ray executor (#13472) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 09:15:32 -08:00
Cody Yu 9206b3d7ec [V1][PP] Run engine busy loop with batch queue (#13064) 2025-02-15 03:59:01 -08:00
Rui Qiao 9605c1256e [V1][core] Implement pipeline parallel on Ray (#12996) 2025-02-13 08:02:46 +00:00
youkaichao 91dd8f7aa6 [bugfix] respect distributed_executor_backend in world_size=1 (#12934) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-08 16:17:08 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus Leung 59a0192fb9 [Core] Interface for accessing model from VllmRunner (#10353) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-20 15:00:59 +08:00
youkaichao 6d0e3d3724 [core] clean up executor class hierarchy between v1 and v0 (#12171) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-18 14:35:15 +08:00
youkaichao 87a0c076af [core] allow callable in collective_rpc (#12151) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-17 20:47:01 +08:00
Chen ZhangandCody Yu 69d765f5a5 [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-17 07:39:35 +00:00
youkaichao bf53e0c70b Support torchrun and SPMD-style offline inference (#12071) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-16 19:58:53 +08:00
Keyun Tong ad388d25a8 Type-fix: make execute_model output type optional (#12020) 2025-01-15 09:44:56 +00:00
youkaichao ad34c0df0f [core] platform agnostic executor via collective_rpc (#11256) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-15 13:45:21 +08:00
youkaichao 89ce62a316 [platform] add ray_device_key (#11948) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-13 16:20:52 +08:00
Rui Qiao 022c5c6944 [V1] Refactor get_executor_cls (#11754) 2025-01-06 07:59:16 +00:00
WangErXiao e5d7ed0c53 [V1] log GPU blocks num for MultiprocExecutor (#11656) 2025-01-04 00:13:12 +00:00
Robert ShawandTyler Michael Smith 1543914c04 [V1] Improve TP>1 Error Handling + Stack Trace (#11721) 
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-01-03 21:29:11 +00:00
Robert Shaw df04dffade [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) 2024-12-28 01:45:08 +00:00
Rui Qiao 9832e5572a [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) 2024-12-24 19:49:46 -08:00
Rui Qiao a491d6f535 [V1] TP Ray executor (#11107) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2024-12-23 23:00:12 +00:00
Tyler Michael Smith 2bfdbf2a36 [V1][Core] Use weakref.finalize instead of atexit (#11242) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-16 22:11:33 -08:00
Mark McLoughlin 6d917d0eeb Enable mypy checking on V1 code (#11105) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2024-12-14 09:54:04 -08:00
Russell Bryant 4863e5fba5 [Core] V1: Use multiprocessing by default (#11074) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-13 16:27:32 -08:00
Russell Bryant ccede2b264 [Core] cleanup zmq ipc sockets on exit (#11115) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-12-11 19:12:24 -08:00
Tyler Michael Smith 28b3a1c7e5 [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-10 06:28:14 +00:00
Robert Shaw 91c9ebbb1b [V1] Fix Configs (#9971) 2024-11-04 00:24:40 +00:00
Robert Shaw d6459b4516 [V1] Fix EngineArgs refactor on V1 (#9954) 2024-11-02 07:44:38 -07:00
youkaichaoandNick Hill e893795443 [2/N] executor pass the complete config to worker/modelrunner (#9938) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2024-11-02 07:35:05 -07:00
Woosuk Kwon 6c5af09b39 [V1] Implement vLLM V1 [1/N] (#9289) 2024-10-22 01:24:07 -07:00