iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
3,987 Commits 3 Branches 0 Tags
07064cb1d49d2b04ec58d8876bee2cd8281eedf5
Commit Graph
181 Commits
Author SHA1 Message Date
youkaichao b12e87f942 [platforms] enable platform plugins (#11602) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 20:24:45 +08:00
Robert Shaw df04dffade [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) 2024-12-28 01:45:08 +00:00
Cyrus Leung 101418096f [VLM] Support caching in merged multi-modal processor (#11396) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-27 17:22:48 +00:00
Robert Shaw 55fb97f7bd [2/N] API Server: Avoid ulimit footgun (#11530) 2024-12-26 23:43:05 +00:00
Cyrus Leung eec906d811 [Misc] Add placeholder module (#11501) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-26 13:12:51 +00:00
Rafael Vasquez 32aa2059ad [Docs] Convert rST to MyST (Markdown) (#11145) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-23 22:35:38 +00:00
youkaichao c954f21ac0 [misc] add early error message for custom ops (#11355) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-19 21:18:25 -08:00
Cyrus Leung cdf22afdda [Misc] Clean up and consolidate LRUCache (#11339) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-20 00:59:32 +08:00
Cyrus Leung 6142ef0ada [VLM] Merged multimodal processor for Qwen2-Audio (#11303) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-19 06:14:17 +00:00
Joe Runde 2d1b9baa8f [Bugfix] Fix request cancellation without polling (#11190) 2024-12-17 12:26:32 -08:00
youkaichao 551603feff [core] overhaul memory profiling and fix backward compatibility (#10511) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-16 13:32:25 -08:00
Cyrus Leung 93abf23a64 [VLM] Fully dynamic prompt replacement in merged input processor (#11199) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-14 17:52:18 +00:00
Chenguang Liandnoemotiovon d1fa714cb1 [Refactor]A simple device-related refactor (#11163) 
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
 2024-12-13 13:39:00 +00:00
Mor Zusman ffa48c9146 [Model] PP support for Mamba-like models (#10992) 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
 2024-12-10 21:53:37 -05:00
Tyler Michael Smith 28b3a1c7e5 [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-12-10 06:28:14 +00:00
xendoandJerzy Zagorski 9c6459e4cb [Neuron] Upgrade neuron to 2.20.2 (#11016) 
Signed-off-by: Jerzy Zagorski <jzagorsk@amazon.com>
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
 2024-12-09 13:53:24 -08:00
youkaichao 1b62745b1d [core][executor] simplify instance id (#10976) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-07 09:33:45 -08:00
Cyrus Leung aa39a8e175 [Doc] Create a new "Usage" section (#10827) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-05 11:19:35 +08:00
Gregory Shtrasberg a061fe601e [Build][Bugfix] Using the correct type hint (#10866) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2024-12-03 15:47:55 -05:00
Cyrus Leung 133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
tomeras91 395b1c7454 [Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635) 
Signed-off-by: Tomer Asida <tomera@ai21.com>
 2024-11-27 13:21:10 -08:00
youkaichao 6e9ff050c8 [misc] do not read HOST_IP (#10644) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-25 17:04:50 -08:00
Cyrus Leung c8acd80548 [2/N] handling placeholders in merged multi-modal processor (#10485) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-22 21:25:09 -08:00
youkaichao eebad39f26 [torch.compile] support all attention backends (#10558) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-22 14:04:42 -08:00
youkaichao 7560ae5caf [8/N] enable cli flag without a space (#10529) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-21 12:30:42 -08:00
Roger Wang c51e397fe8 [Misc] Suppress duplicated logging regarding multimodal input pipeline (#10530) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-21 09:21:31 -08:00
Luka Govedič 8b0fe06c89 [torch.compile] Inductor code caching fix (#10273) 
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
 2024-11-20 21:44:57 -08:00
Yanyi Liu 9e05252b46 [Misc] Add __setitem__ for LazyDict (#10469) 
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
 2024-11-20 04:44:57 +00:00
youkaichao 51bb12d17b [4/N][torch.compile] clean up set_torch_compile_backend (#10401) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-17 23:57:20 -08:00
Cyrus Leung b311efd0bd [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 09:34:17 +00:00
Shawn Duandyoukaichao 20cf2f553c [Misc] small fixes to function tracing file path (#9543) 
Signed-off-by: Shawn Du <shawnd200@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-10 15:21:06 -08:00
Cyrus Leung db7db4aab9 [Misc] Consolidate ModelConfig code related to HF config (#10104) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-07 06:00:21 +00:00
+26
Konrad ZaworaKunshang JiSanju C SudhakaranMichal AdamczykMarceli FylcekHimangshu LahkarVivek GoelyuwenzhoDominika Olszewskabarak goldbergMichal SzutenbergJan KanieckiAgata DobrzyniewiczKrzysztof WisniewskiDudi LesterIlia TarabanChendi.XueMichał KuligowskiJakub MaksymczukTomasz ZielinskiSun ChoiIryna BoikoBob Zhuhlin99Zehao HuangAndrzej KotłowskiYan TomsinskyNir DavidYu-ZhouRuheena Suhani ShaikKarol DamaszkeMarcin SwiniarskiWoosuk KwonJacek CzajaJacek CzajaYuan
 		a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>
 2024-11-06 01:09:10 -08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
Chauncey 93dee88f6b [Misc] vllm CLI flags should be ordered for better user readability (#10017) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2024-11-05 18:59:56 +08:00
youkaichao af7380d83b [torch.compile] fix cpu broken code (#9947) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 23:35:47 -07:00
sroy745 a78dd3303e [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) 2024-11-01 23:22:49 -07:00
youkaichao 96e0c9cbbd [torch.compile] directly register custom op (#9896) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 21:56:09 -07:00
Alex Brooks cc98f1e079 [CI/Build] VLM Test Consolidation (#9372) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-30 09:32:17 -07:00
youkaichao ff5ed6e1bc [torch.compile] rework compile control with piecewise cudagraph (#9715) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-29 23:03:49 -07:00
wangshuai09 622b7ab955 [Hardware] using current_platform.seed_everything (#9785) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-29 14:47:44 +00:00
wangshuai09 4e2d95e372 [Hardware][ROCM] using current_platform.is_rocm (#9642) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-28 04:07:00 +00:00
madt2709 34a9941620 [Bugfix] Fix load config when using bools (#9533) 2024-10-27 13:46:41 -04:00
youkaichao 8549c82660 [core] cudagraph output with tensor weak reference (#9724) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-27 00:19:28 -07:00
Mengqing Cao 5cbdccd151 [Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716) 2024-10-26 10:59:06 +00:00
Mengqing Cao 2394962d70 [Hardware][XPU] using current_platform.is_xpu (#9605) 2024-10-23 08:28:21 +00:00
xendoandJerzy Zagorski 9dbcce84a7 [Neuron] [Bugfix] Fix neuron startup (#9374) 
Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>
 2024-10-22 12:51:41 +00:00
wangshuai09 3ddbe25502 [Hardware][CPU] using current_platform.is_cpu (#9536) 2024-10-22 00:50:43 -07:00
Travis JohnsonandCyrus Leung b729901139 [Bugfix]: serialize config by value for --trust-remote-code (#6751) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-10-21 19:46:24 -07:00
Nick Hill 9d9186be97 [Frontend] Reduce frequency of client cancellation checking (#7959) 2024-10-21 13:28:10 -07:00