iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,797 Commits 3 Branches 0 Tags
4adc66f64d56338489d00d94de6e13d95741c4be
Commit Graph
83 Commits
Author SHA1 Message Date
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Congcong Chen 2c11a738b3 [Model] New model support for microsoft/Phi-4-mini-flash-reasoning (#20702) 
Signed-off-by: Congcong Chen <congcongchen@microsoft.com>
 2025-07-12 06:02:10 -07:00
Kunshang Ji 0b407479ef [misc]refactor Platform.set_device method (#20262) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-09 01:39:47 +00:00
Yang YangandKunshang Ji 6e2c19ce22 [Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU (#19410) 
Signed-off-by: dbyoung18 <yang5.yang@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-07 04:32:32 +00:00
Gabriel Marinho a4113b035c [Platform] Add custom default max tokens (#18557) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
 2025-07-04 10:50:17 +08:00
Seiji Eicher 65397e40f5 [Bugfix] Allow CUDA_VISIBLE_DEVICES='' in Platform.device_id_to_physical_device_id (#18979) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-06-26 00:01:57 -07:00
kourosh hakhamaneshi 5e666f72cd [Bugfix][Ray] Set the cuda context eagerly in the ray worker (#19583) 2025-06-19 22:01:16 -07:00
wangxiyuan 257ab95439 [Platform] Allow platform use V1 Engine by default (#19792) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-06-18 13:03:36 +00:00
Driss Guessous cf02f9b283 Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
Michael Goin 87360308b7 [V1] Use FlashInfer by default on Blackwell GPUs (#19118) 2025-06-05 15:40:39 -04:00
Nicolò Lucchesi 9ef9173cfa [P/D][NixlConnector] Enable FlashInfer backend (#19090) 2025-06-05 17:10:15 +00:00
Kaixi Hou 41aa578428 [NVIDIA] Add Cutlass MLA backend (#17625) 2025-06-03 21:40:26 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Nicolò Lucchesi 4b7817c119 [Misc] Add missing _Backend enums (#19081) 
Signed-off-by: nicklucche <nlucches@redhat.com>
 2025-06-03 16:15:16 +00:00
Mengqing Cao d781930f90 [Platform][Dist] Make torch distributed process group extendable (#18763) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-28 10:52:34 +00:00
Hyogeun Oh (오효근) a68e293cb9 [Doc] Convert Sphinx directives ( {class}, {meth}, {attr}, ...) to MkDocs format for better documentation linking (#18663) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-05-27 01:44:20 -07:00
Harry Mellor 2edb533af2 Replace {func} with mkdocs style links (#18610) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 05:51:38 -07:00
Mengqing Caoandyoukaichao f8d2cc5f55 [Compile][Platform] Make PiecewiseBackend pluggable and extendable (#18076) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-05-22 12:11:53 -07:00
Harry Mellor c8ea982d9b Update deprecated type hinting in platform, plugins, triton_utils, vllm_flash_attn (#18129) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 05:28:16 -07:00
Tao He 60f7624334 Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844) 2025-05-12 19:52:47 -07:00
Jade Zheng 289199feb6 [Core] Use platform-agnostic device control for DP engine core (#17245) 
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
 2025-05-12 12:09:16 -07:00
Isotr0py 5c4c08f6f1 [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-09 17:16:12 +00:00
vllmellmqli88Hongxia Yang
 		3c9396a64f [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
 2025-05-09 10:42:05 +08:00
Akshat TripathiandChengji Yao c20ef40fd0 [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-05-07 16:28:47 -04:00
Bowen Bao db593aa67f [Quantization] Quark MXFP4 format loading (#16943) 2025-05-07 15:05:05 -04:00
Harry Mellor d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Gregory Shtrasberg a92842454c [Bugfix][ROCm] Using device_type because on ROCm the API is still torch.cuda (#17601) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-02 22:25:47 -07:00
Nicolò Lucchesi 889ebb2638 [Misc] Minor typo/grammar in platforms/interface.py (#17307) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-04-28 05:45:42 -07:00
vllmellmandqli88 30bc3e0f66 [FEAT][ROCm]: Support AITER MLA (#15893) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
 2025-04-22 09:31:13 -07:00
Charlie FuandTyler Michael Smith 188b7f9b8c [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) 
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-04-21 20:46:22 -07:00
Shanshan Shen 7272bfae77 [Misc] Refactor platform to get device specific stream and event (#14411) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-04-21 21:25:49 +08:00
Joe Runde e1b004839a [Hardware] Add processor inputs to platform validation (#16680) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-04-16 09:28:42 -07:00
Shuqiao Li d2020acac7 config check sleep mode support oot platforms (#16562) 2025-04-14 16:31:50 -07:00
Joe Runde cb391d85dc [Hardware] add platform-specific request validation api (#16291) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-04-09 12:50:01 -07:00
yihong 04149cce27 [BugFix] fix some typos found by typos. (#16314) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-09 03:43:59 -07:00
Shanshan Shen e9ba99f296 [V1][Structured Output] Add supports_structured_output() method to Platform (#16148) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-04-07 11:06:24 +00:00
Ilya Markovandilmarkov ef608c37a7 [Distributed] [ROCM] Fix custom allreduce enable checks (#16010) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-04-04 09:39:08 -07:00
Joe Runde 5f063a80bd [bugfix] add supports_v1 platform interface (#15417) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-03-25 15:00:32 -04:00
Russell Bryant b877031d80 Remove openvino support in favor of external plugin (#15339) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-22 14:06:39 -07:00
Isotr0pyandWoosuk Kwon f8a08cb90d [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-21 03:14:19 +00:00
Richard Liu a8f12a63fd Fix env vars for running Ray distributed backend on GKE (#15166) 
Signed-off-by: Richard Liu <ricliu@google.com>
 2025-03-20 14:59:33 +00:00
Jeff Daily a1c8f3796c dynamic distpatch of fp8 kernels (#14245) 
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
 2025-03-11 10:54:56 -04:00
Mengqing Cao b87c21fc89 [Misc][Platform] Move use allgather to platform (#14010) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-03 15:40:04 +08:00
Lucas WilkinsonandYang Chen 2e94b9cfbb [Attention] Flash MLA for V1 (#13867) 
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yang Chen <yangche@fb.com>
 2025-02-27 23:03:41 +00:00
Yang Chen 58d1b2aa77 [Attention] MLA support for V1 (#13789) 
Signed-off-by: Yang Chen <yangche@fb.com>
 2025-02-27 13:14:17 -05:00
Lucas Wilkinson f95903909f [Kernel] FlashMLA integration (#13747) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-02-27 10:35:08 +08:00
Mengqing Cao 238dfc8ac3 [MISC] tiny fixes (#13378) 2025-02-17 00:57:13 -08:00
youkaichao a0231b7c25 [platform] add base class for communicators (#13208) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-16 22:14:22 +08:00
Alexander Matveev 45f90bcbba [WIP] TPU V1 Support Refactored (#13049) 2025-02-14 00:21:53 -08:00
wangxiyuan 2e3b969ec0 [Platform] add pre_register_and_update function (#12432) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-02-11 22:06:46 +08:00