iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
4,376 Commits 3 Branches 0 Tags
415f19474dedc69934cab79cfb8f5bdc19e2ae0d
Commit Graph
75 Commits
Author SHA1 Message Date
Harry Mellor 823ab79633 Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
youkaichao 310aca88c9 [perf]fix current stream (#11870) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-09 07:18:21 +00:00
XiaobingZhang e512f76a89 fix init error for MessageQueue when n_local_reader is zero (#11768) 2025-01-07 06:12:48 +00:00
cennn 9e764e7b10 [distributed] remove pynccl's redundant change_state (#11749) 2025-01-06 09:05:48 +08:00
cennn 635b897246 [distributed] remove pynccl's redundant stream (#11744) 2025-01-05 23:09:11 +08:00
youkaichao 62de37a38e [core][distributed] initialization from StatelessProcessGroup (#10986) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-12 09:04:19 +00:00
Tyler Michael Smith 28b3a1c7e5 [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-10 06:28:14 +00:00
youkaichao 21fe7b481a [core][distributed] add pynccl broadcast (#10843) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-03 04:53:23 +00:00
Sage Mooreandyoukaichao 9a88f89799 custom allreduce + torch.compile (#10121) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 22:00:16 -08:00
Tyler Michael Smith 978b39744b [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) 2024-11-22 22:14:03 -05:00
youkaichao e6de9784d2 [core][distributed] add stateless process group (#10216) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-11 09:02:14 -08:00
Yan Ma f10797c0ce [Bugfix][XPU] Fix xpu tp by introducing XpuCommunicator (#10144) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-08 09:41:03 +00:00
Hanzhi Zhou 6192e9b8fe [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) 
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
 2024-11-06 23:50:47 -08:00
+26
Konrad ZaworaKunshang JiSanju C SudhakaranMichal AdamczykMarceli FylcekHimangshu LahkarVivek GoelyuwenzhoDominika Olszewskabarak goldbergMichal SzutenbergJan KanieckiAgata DobrzyniewiczKrzysztof WisniewskiDudi LesterIlia TarabanChendi.XueMichał KuligowskiJakub MaksymczukTomasz ZielinskiSun ChoiIryna BoikoBob Zhuhlin99Zehao HuangAndrzej KotłowskiYan TomsinskyNir DavidYu-ZhouRuheena Suhani ShaikKarol DamaszkeMarcin SwiniarskiWoosuk KwonJacek CzajaJacek CzajaYuan
 		a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>
 2024-11-06 01:09:10 -08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
youkaichao 4be3a45158 [distributed] add function to create ipc buffers directly (#10064) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-05 22:35:03 -08:00
Tyler Michael Smith 04bbf38e05 [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-05 01:08:21 +00:00
youkaichao 1ab6f6b4ad [core][distributed] fix custom allreduce in pytorch 2.5 (#9815) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-29 17:06:24 -07:00
youkaichao 663874e048 [torch.compile] improve allreduce registration (#9061) 2024-10-04 16:43:50 -07:00
youkaichao 18e60d7d13 [misc][distributed] add VLLM_SKIP_P2P_CHECK flag (#8911) 2024-09-27 14:27:56 -07:00
Russell Bryant b05f5c9238 [Core] Allow IPv6 in VLLM_HOST_IP with zmq (#8575) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-09-23 12:15:41 -07:00
Russell Bryant d65798f78c [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-09-18 16:10:27 +00:00
youkaichao 99aa4eddaf [torch.compile] register allreduce operations as custom ops (#8526) 2024-09-16 22:57:57 -07:00
Richard Liu 2148441fd3 [TPU] Support single and multi-host TPUs on GKE (#7613) 2024-08-30 00:27:40 -07:00
youkaichao 05826c887b [misc] fix custom allreduce p2p cache file generation (#7853) 2024-08-26 15:02:25 -07:00
youkaichaoandcjackal d95cc0a55c [core][misc] update libcudart finding (#7620) 
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2024-08-16 23:01:35 -07:00
bnellnm e680349994 [Bugfix] Fix custom_ar support check (#7617) 2024-08-16 19:05:49 -07:00
Woosuk Kwon 59edd0f134 [Bugfix][CI] Import ray under guard (#7486) 2024-08-13 17:12:58 -07:00
Woosuk Kwon a08df8322e [TPU] Support multi-host inference (#7457) 2024-08-13 16:31:20 -07:00
youkaichao 639159b2a6 [distributed][misc] add specialized method for cuda platform (#7249) 2024-08-07 08:54:52 -07:00
youkaichao 16a1cc9bb2 [misc][distributed] improve libcudart.so finding (#7127) 2024-08-04 11:31:51 -07:00
Woosuk Kwon fad5576c58 [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) 2024-07-27 10:28:33 -07:00
Woosuk Kwon d09b94ca58 [TPU] Support collective communications in XLA devices (#6813) 2024-07-27 01:45:57 +00:00
youkaichao 740374d456 [core][distributed] fix zmq hang (#6759) 2024-07-24 17:37:12 -07:00
Woosuk Kwon b6df37f943 [Misc] Remove abused noqa (#6619) 2024-07-21 23:47:04 +08:00
youkaichao 07eb6f19f3 [bugfix][distributed] fix multi-node bug for shared memory (#6597) 2024-07-19 15:34:34 -07:00
Nick Hill d25877dd9b [BugFix] Avoid secondary error in ShmRingBuffer destructor (#6530) 2024-07-17 22:24:43 -07:00
youkaichao 7f62077af5 [misc][distributed] improve tests (#6488) 2024-07-16 17:35:52 -07:00
Cyrus Leung d97011512e [CI/Build] vLLM cache directory for images (#6444) 2024-07-15 23:12:25 -07:00
youkaichao 2b0fb53481 [distributed][misc] be consistent with pytorch for libcudart.so (#6346) 
[distributed][misc] keep consistent with how pytorch finds libcudart.so (#6346)
 2024-07-11 19:35:17 -07:00
youkaichao da78caecfa [core][distributed] zmq fallback for broadcasting large objects (#6183) 
[core][distributed] add zmq fallback for broadcasting large objects (#6183)
 2024-07-09 18:49:11 -07:00
youkaichao 614aa51203 [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) 2024-06-30 20:07:34 -07:00
Cyrus Leung 99397da534 [CI/Build] Add TP test for vision models (#5892) 2024-06-29 15:45:54 +00:00
youkaichao 515080ad2f [bugfix][distributed] fix shm broadcast when the queue size is full (#5801) 2024-06-25 21:56:02 -07:00
Matt Wong dd793d1de5 [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) 2024-06-25 15:56:15 -07:00
Murali Andoorveedu 5d4d90536f [Distributed] Add send and recv helpers (#5719) 2024-06-23 14:42:28 -07:00
youkaichao 832ea88fcb [core][distributed] improve shared memory broadcast (#5754) 2024-06-22 10:00:43 -07:00
youkaichaoandCody Yu d9a252bc8e [Core][Distributed] add shm broadcast (#5399) 
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
 2024-06-21 05:12:35 +00:00
youkaichao 6c5b7af152 [distributed][misc] use fork by default for mp (#5669) 2024-06-20 17:06:34 -07:00
youkaichao db5ec52ad7 [bugfix][distributed] improve p2p capability test (#5612) 
[bugfix][distributed] do not error if two processes do not agree on p2p capability (#5612)
 2024-06-18 07:21:05 +00:00