iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
9,510 Commits 3 Branches 0 Tags
d593cf28fa020dbae53dae19122aac8aeeeae0bc
Commit Graph
28 Commits
Author SHA1 Message Date
Woosuk Kwon 105d3d62ef [TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 01:12:36 -07:00
22quinnNick HillWoosuk Kwon
 		f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
Syed Muhammad Bin Asif 609b533cb6 [Bugfix] Add proper comparison for package versions (#22314) 
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
 2025-08-06 20:31:03 -07:00
Michael Goin 34a20c49b3 [Logs] Change flashinfer sampler logs to once (#21759) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-28 06:59:51 -07:00
Mengqing Cao f3a683b7c9 [Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-07-25 05:53:07 -07:00
Lu Fang 8d0a01a5f2 [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-07-21 13:47:47 -07:00
Isotr0py 32c9be2200 [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
afeldman-nmandNick Hill 48fb076cbc [V1] LogitsProcessor programming model (#16728) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-02 09:10:42 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Lukas Geiger e7523c2e03 [V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608) 2025-05-26 11:49:36 -04:00
Bowen Wangandmgoin 7fdfa01530 [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-05-16 15:14:03 -07:00
Nicolò LucchesiandHyesoo Yang eb5819b2d9 [V1][TPU] Enable Top K (#15489) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>
 2025-04-17 18:18:11 +00:00
Hyesoo Yangandroot 1b84eff03a [V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
 2025-04-02 17:18:08 -07:00
Brayden Zhong 6efb195a6e [V1] Fix: make sure k_index is int64 for apply_top_k_only (#15907) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-04-01 19:06:44 -07:00
Nick Hill 35fad35a48 [V1][Sampler] Faster top-k only implementation (#15478) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-26 10:56:47 -07:00
Woosuk Kwon 911c8eb000 [Minor][Spec Decode] Remove compiled_softmax (#15416) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-24 19:09:04 -07:00
Hyesoo Yang 47195057e9 [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) 
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
 2025-03-20 19:19:40 -07:00
Nicolò Lucchesi d8c6d7d6b5 [V1][TPU] Support V1 Sampler for ragged attention (#14227) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-03-19 21:00:39 -07:00
Woosuk Kwon 99abb8b650 [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-18 14:31:54 -07:00
DefTruth acaea3bb07 [Bugfix][V1] Fix flashinfer sampling (#14815) 2025-03-14 20:42:38 -07:00
Woosuk Kwon 32ef4983cd [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-13 20:40:35 -07:00
22quinnandNick Hill eb8b5eb183 [V1] Support bad_words in sampler (#13376) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-08 14:50:26 -08:00
Harry Mellor cf069aa8aa Update deprecated Python 3.8 typing (#13971) 2025-03-02 17:34:51 -08:00
Nick Hill 30172b4947 [V1] Optimize handling of sampling metadata and req_ids list (#13244) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-18 12:15:33 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Kazuhiro Serizawa 6d70198b17 [Doc] Fix typo (#11666) 
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
 2025-01-01 08:10:10 +00:00
Woosuk Kwon 81b979f2a8 [V1] Fix yapf (#11538) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-27 09:47:10 +09:00
Woosuk Kwon 371d04d39b [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-27 09:32:38 +09:00