iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,486 Commits 3 Branches 0 Tags
de92d916fe8a897b00a8adb0aab9ed9ec99f2b6c
Commit Graph
94 Commits
Author SHA1 Message Date
Michael Goin e66d787bce Disable FlashInfer sampler by default (#26859) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-15 02:35:18 +00:00
ihb2032 4a61950f4d [Hardware][CPU] Disable torch.compile for RISC-V to prevent APIError (#26693) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn
 2025-10-13 07:56:01 -07:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Nick Hill ddcbc2f334 [Misc] Misc code simplifications (#26450) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-09 02:10:06 -07:00
Harry Mellor e09d1753ec Remove Python 3.9 support ahead of PyTorch 2.9 in v0.11.1 (#26416) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 10:40:42 -07:00
Harry Mellor 2f99f2f506 Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Sergei SkvortsovSergei SkvortsovNick Hill
 		6ebaf43ee4 [V1] Logit processors for rejection sampler (#19482) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Signed-off-by: Sergei Skvortsov <yvorott@gmail.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-07 13:02:49 -07:00
Harry Mellor b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Corey Lowman 0879736aab [Perf] Remove hardcoded num_warps=1 (#26183) 
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
 2025-10-03 20:38:50 +00:00
Ekagra RanjanandNick Hill e71b8e210d [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-09-25 15:22:03 -07:00
Russell Bryant 532a6cfccb [ux] Switch a warning to debug about a pytorch fallback (#23750) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-25 14:38:16 +00:00
Li, Jiang eb32335e35 [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-25 13:29:11 +00:00
courage17340 a676e668ee [Bugfix] fix apply_temperature to avoid nan in probs (#24734) 
Signed-off-by: courage17340 <courage17340@163.com>
 2025-09-25 05:32:21 +00:00
Wenlong Wang 032d661d27 [Docs] Fix warnings in mkdocs build (continued) (#25042) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-20 11:45:18 +00:00
Harry Mellor aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Andrew SansomandCyrus Leung 9a4600e4dc [CORE] Prompt Embeddings Support for v1 Engine (#24278) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-19 08:03:09 +08:00
afeldman-nmandJoseph Marinier 7ae9887542 [V1] Logits processor docs (#22919) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Joseph Marinier <Joseph.Marinier@gmail.com>
 2025-09-17 11:53:12 -07:00
co63ocandRussell Bryant 3144d90217 fix some typos (#24167) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-09-10 06:21:23 -07:00
Woosuk Kwon 105d3d62ef [TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 01:12:36 -07:00
afeldman-nm 136d853e65 [V1] Wrapper which plumbs request-level logits processors into vLLM batch-level logits processing (#23656) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
 2025-09-03 02:52:51 +00:00
Jingkai He 57d4ede520 [bugfix] [spec-decoding] fix data race in sample_recovered_tokens_kernel (vLLM v1) (#23829) 
Signed-off-by: He-Jingkai <he-jingkai@outlook.com>
 2025-08-28 19:05:20 +00:00
Woosuk Kwon a3432f18fd [BugFix][Spec Decode] Use float64 for uniform_probs (#23803) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-28 12:26:45 +00:00
Hyogeun Oh (오효근) 4e4d017b6f [Docs] Fix warnings in mkdocs build (continued) (#23743) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
 2025-08-27 17:17:29 +00:00
Nick Hill 3ce8285d6d [LogitsProcs] Deduplicate built-in LP implementation logic (#23362) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-27 23:11:33 +08:00
Nick Hill add1adfec7 [BugFix] Fix MinPLogitsProcessor.update_states() (#23401) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-23 08:22:11 +08:00
Nick Hill c80c53a30f [BugFix] Fix batch updates for pooling models (#23398) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-23 08:20:41 +08:00
Nick Hill 603fbbbce0 [Misc] Misc code cleanup/simplification (#23304) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-21 17:22:55 +00:00
22quinnNick HillWoosuk Kwon
 		f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
afeldman-nmNick HillAndrew FeldmanHarry Mellor
 		bf7f470b22 [V1] Logits processors extensibility (#19912) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-16 12:59:17 -07:00
Syed Muhammad Bin Asif 609b533cb6 [Bugfix] Add proper comparison for package versions (#22314) 
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
 2025-08-06 20:31:03 -07:00
Michael Goin 34a20c49b3 [Logs] Change flashinfer sampler logs to once (#21759) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-28 06:59:51 -07:00
Mengqing Cao f3a683b7c9 [Bugfix][Logprobs] Fix logprobs op to support more backend (#21591) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-07-25 05:53:07 -07:00
Lu Fang accac82928 [Sampler] Introduce logprobs mode for logging (#21398) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-07-23 01:39:25 -07:00
Jialin Ouyang a32237665d [Core] Optimize update checks in LogitsProcessor (#21245) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-07-22 05:27:18 -07:00
Lu Fang 8d0a01a5f2 [v1][sampler] Inplace logprobs comparison to get the token rank (#21283) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-07-21 13:47:47 -07:00
Maximilien de Bayser 5de8d9f111 Remove extra tensor on CPU (#20693) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-07-12 14:06:34 +08:00
Isotr0py 32c9be2200 [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
Jee Jee Li 1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
afeldman-nmandNick Hill 48fb076cbc [V1] LogitsProcessor programming model (#16728) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-02 09:10:42 -07:00
afeldman-nm 19a53b2783 [V1] Decouple GPU and TPU InputBatch (#19778) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
 2025-06-18 06:38:13 +00:00
Yu Guo 8267f9916f improve logits bias (#19041) 2025-06-06 19:59:25 +08:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Lukas Geiger e7523c2e03 [V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608) 2025-05-26 11:49:36 -04:00
Bowen Wangandmgoin 7fdfa01530 [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-05-16 15:14:03 -07:00
Mengqing Cao f9bc5a0693 [Bugfix] Fix triton import with local TritonPlaceholder (#17446) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-06 17:53:09 +08:00
Nicolò Lucchesi 5941e0b7ea [TPU][V1] Add support for top-logprobs (#17072) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-05-05 14:20:15 -07:00
Harry Mellor d6484ef3c3 Add full API docs and improve the UX of navigating them (#17485) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-03 19:42:43 -07:00
Woosuk Kwon 41fb013d29 [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-23 14:57:43 -07:00
Nicolò Lucchesiandmgoin fa3bba2a53 [TPU][V1] Enable Top-P (#16843) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-04-22 00:46:07 +00:00