iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
54 Commits
Author SHA1 Message Date
Kunshang Ji 8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00
Kunshang Ji 92510edc32 remove cuda check in top_k_top_p_triton kernel (#35011) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-24 22:22:31 -08:00
Xiao Li 30132cd144 Fix apply_top_k_top_p_triton called by non-cuda logits Tensor (#35030) 
Signed-off-by: Xiao Li <ilx@meta.com>
 2026-02-21 21:11:54 -08:00
Jongseok ParkSunga KimNick Hill
 		c656ba3b4d [Kernel] Triton-based Top-k and Top-p sampler kernels (#33538) 
Signed-off-by: js_park <cakeng@naver.com>
Signed-off-by: Jongseok Park <37990712+cakeng@users.noreply.github.com>
Signed-off-by: Sunga Kim <sunga.kim@berkeley.edu>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Sunga Kim <sunga.kim@berkeley.edu>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-02-17 23:14:30 +00:00
Pleaplusone 77c16df31d [ROCm][Bugfix] Disable hip sampler to fix deepseek's accuracy issue on ROCm (#32413) 
Signed-off-by: ganyi <ygan@amd.com>
 2026-01-15 16:35:47 +00:00
Nick Hill 10ef65eded [BugFix] Fix bad words with speculative decoding (#31908) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-07 15:46:42 -05:00
Harry MellorandCyrus Leung 9eec282cb5 Guard FlashInfer sampler using the same check as FlashInfer attention backend (#29415) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-28 08:34:48 -08:00
Xiao Li ed6ae1e36a [AITER] [ROCm] Fix crash when loading llama4 model with old aiter version installed, fallback to forward_native implementation (#29124) 
Signed-off-by: Xiao Li <ilx@meta.com>
 2025-11-20 17:54:35 -08:00
vllmellm 0af3d4f0df [FEAT] [AITER] [ROCm] integrate aiter sampling ops (#26084) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-11-18 17:28:34 +00:00
Zhang Xiangze 7bdb42b2f2 [CPU]Avoid repeated random sample compile (#28260) 
Signed-off-by: Zhang Xiangze <Xiangze.Zhang@arm.com>
 2025-11-07 11:03:57 +00:00
Nick Hill c2ed069b32 [BugFix] Fix mixed penalties batch with async scheduling (#27910) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 10:51:24 -07:00
Wentao Ye 52efc34ebf [Log] Optimize Startup Log (#26740) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-24 19:27:04 -04:00
Jonathan Chen ca76486a16 [Chore] Separate out vllm.utils.platform_utils.py (#27374) 
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
 2025-10-23 19:08:06 +00:00
Isotr0py 6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Jee Jee Li fec2b341ad [Kernel] Lazy import FlashInfer (#26977) 2025-10-17 04:48:18 +00:00
Akash kaothalkarandAkash Kaothalkar f7d318de2b [Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
 2025-10-15 22:36:59 -07:00
Michael Goin e66d787bce Disable FlashInfer sampler by default (#26859) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-15 02:35:18 +00:00
ihb2032 4a61950f4d [Hardware][CPU] Disable torch.compile for RISC-V to prevent APIError (#26693) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn
 2025-10-13 07:56:01 -07:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor 2f99f2f506 Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Sergei SkvortsovSergei SkvortsovNick Hill
 		6ebaf43ee4 [V1] Logit processors for rejection sampler (#19482) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Signed-off-by: Sergei Skvortsov <yvorott@gmail.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-07 13:02:49 -07:00
Harry Mellor b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Russell Bryant 532a6cfccb [ux] Switch a warning to debug about a pytorch fallback (#23750) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-25 14:38:16 +00:00
Li, Jiang eb32335e35 [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-25 13:29:11 +00:00
Harry Mellor aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Woosuk Kwon 105d3d62ef [TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 01:12:36 -07:00
22quinnNick HillWoosuk Kwon
 		f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
Syed Muhammad Bin Asif 609b533cb6 [Bugfix] Add proper comparison for package versions (#22314) 
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
 2025-08-06 20:31:03 -07:00
Michael Goin 34a20c49b3 [Logs] Change flashinfer sampler logs to once (#21759) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-28 06:59:51 -07:00
Mengqing Cao f3a683b7c9 [Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-07-25 05:53:07 -07:00
Lu Fang 8d0a01a5f2 [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-07-21 13:47:47 -07:00
Isotr0py 32c9be2200 [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
afeldman-nmandNick Hill 48fb076cbc [V1] LogitsProcessor programming model (#16728) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-02 09:10:42 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Lukas Geiger e7523c2e03 [V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608) 2025-05-26 11:49:36 -04:00
Bowen Wangandmgoin 7fdfa01530 [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-05-16 15:14:03 -07:00
Nicolò LucchesiandHyesoo Yang eb5819b2d9 [V1][TPU] Enable Top K (#15489) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>
 2025-04-17 18:18:11 +00:00
Hyesoo Yangandroot 1b84eff03a [V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
 2025-04-02 17:18:08 -07:00
Brayden Zhong 6efb195a6e [V1] Fix: make sure k_index is int64 for apply_top_k_only (#15907) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-04-01 19:06:44 -07:00
Nick Hill 35fad35a48 [V1][Sampler] Faster top-k only implementation (#15478) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-26 10:56:47 -07:00
Woosuk Kwon 911c8eb000 [Minor][Spec Decode] Remove compiled_softmax (#15416) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-24 19:09:04 -07:00
Hyesoo Yang 47195057e9 [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
 2025-03-20 19:19:40 -07:00
Nicolò Lucchesi d8c6d7d6b5 [V1][TPU] Support V1 Sampler for ragged attention (#14227) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-03-19 21:00:39 -07:00
Woosuk Kwon 99abb8b650 [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-18 14:31:54 -07:00
DefTruth acaea3bb07 [Bugfix][V1] Fix flashinfer sampling (#14815) 2025-03-14 20:42:38 -07:00
Woosuk Kwon 32ef4983cd [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-13 20:40:35 -07:00
22quinnandNick Hill eb8b5eb183 [V1] Support bad_words in sampler (#13376) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-08 14:50:26 -08:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Nick Hill 30172b4947 [V1] Optimize handling of sampling metadata and req_ids list (#13244) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-18 12:15:33 -08:00