 Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bf95f58d10
|
[Core] Triton kernel for small-batch top-p only masking (#54651)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-03 07:52:47 -07:00 |
|
|
MKQuantum
|
2f41c894e3
|
Fix seed loss when batch contains unseeded requests (#51866)
Signed-off-by: MKQuantum <121593006+MKQuantum@users.noreply.github.com>
|
2026-08-20 22:38:53 +00:00 |
|
|
wangxiyuan
|
cba06764d7
|
[Platform] Fill in the missing backend parameter for torch.compile (#51781)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2026-08-19 10:31:19 +00:00 |
|
|
Nick HillandClaude Opus 4.8
|
615d4cfade
|
[Core] Check for GPU<->CPU syncs during CI (#43107)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 17:42:05 -07:00 |
|
|
|
405bc86768
|
[Perf] Launch the top-k/top-p Triton sampler kernel with 8 warps (#51507)
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-10 10:55:35 -07:00 |
|
|
+1
|
4f1da84eb5
|
Enable gfx1250 ROCm architecture (#46516)
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-31 01:41:12 +00:00 |
|
|
Michael GoinandNick Hill
|
ac36a7a1e7
|
[MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-23 18:13:13 +08:00 |
|
|
vllm-agentandvllm-agent
|
1134545b6f
|
Revert "[Sampler] Stop upcasting logits to fp32 in apply_sampling_params" (#48641) (#49033)
Co-authored-by: vllm-agent <vllm-agent@users.noreply.github.com>
|
2026-07-21 09:36:45 +01:00 |
|
|
|
425c4eafb0
|
[Sampler] Stop upcasting logits to fp32 in apply_sampling_params (#48641)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-17 18:15:20 -07:00 |
|
|
|
3c43237233
|
[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560)
Signed-off-by: xiaojun.wei <jessiewei747@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-06-24 11:00:57 -07:00 |
|
|
Nick Hill
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
|
Andreas Karatzas
|
82a42234be
|
[ROCm][CI] Defer AITER sampler import and isolate server test PYTHONPATH (#44823)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-10 08:56:11 +00:00 |
|
|
Chaojun Zhang
|
fa662b1a8b
|
[XPU] Cap topk/topp Triton BLOCK_SIZE to 4096 to fix Top-p mask difference failures (#44470)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-06-08 09:36:51 +00:00 |
|
|
|
7fe7800fa4
|
[BUG] Fix FP64 Gumbel precision coverage (#43150)
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-05 19:04:14 +08:00 |
|
|
Woosuk Kwon
|
6bad553f4e
|
[Minor] Remove FlashInfer version check in topk_topp_sampler (#44442)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-03 21:06:00 +00:00 |
|
|
Nick Hill
|
ec8d60bea8
|
[Model Runner V2] Use FlashInfer sampler (#42472)
|
2026-06-03 07:59:31 -07:00 |
|
|
Jongseok ParkandNick Hill
|
ca17b6b17d
|
[Perf] Apply single-pass min_larger finding and binary search in Triton Top-p path. (#42191)
Signed-off-by: js_park <cakeng@naver.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-02 17:57:26 -07:00 |
|
|
Tianmu Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
94d3f4d205
|
[CPU Backend] CPU top-k and top-p sampling kernels using Triton (#43633)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-29 15:02:39 +08:00 |
|
|
JohnQinAMDandTJian
|
ed7fe831da
|
[ROCm] Enable the aiter top-k/top-p sampler by default (#43331)
Signed-off-by: John Qin <yanyuan.qin@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-28 13:19:59 -05:00 |
|
|
|
71d810bbf4
|
[XPU] Ensure RNG offset alignment with PyTorch requirements in XPU sampler (#43028)
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-26 02:01:30 +00:00 |
|
|
Xiaogang Zhouandxiaogang.zhou
|
d19db10974
|
[Bugfix] Fix native Triton top-k/top-p kernel assumes contiguous logi… (#42739)
Signed-off-by: xiaogang.zhou <xiaogang.zhou@bytedance.com>
Co-authored-by: xiaogang.zhou <xiaogang.zhou@bytedance.com>
|
2026-05-22 22:56:16 -07:00 |
|
|
Ofir Zafrir
|
8f16c4a5c0
|
[BugFix][CPU][Spec Decode] Fix Eagle implementation on CPU backend (#42468)
Signed-off-by: Ofir Zafrir <ofir.zafrir@intel.com>
|
2026-05-19 03:16:07 +00:00 |
|
|
|
bc03f280c8
|
[XPU] keep generator state of sycl kernel align with pytorch (#41771)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-12 11:44:47 +00:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbee532988
|
[Perf][1/n] Eliminate various GPU<->CPU syncs (#41429)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 20:36:03 +00:00 |
|
|
Kunshang Ji
|
2ceea42958
|
[XPU] use xpu topk topp sample kernel (#39285)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-05 18:05:17 +08:00 |
|
|
Artem Perevedentsev
|
b92ef9ec5a
|
[Perf] Enable FlashInfer top-k/top-p sampler by default (#40376)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-04-29 19:10:34 +04:00 |
|
|
Lucas KabelaandLuka Govedič
|
a8c6ee9b78
|
[Performance Improvement] Update batched_count_greater_than to handle batch size 1 without recompile (#38933)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
|
2026-04-09 23:51:31 +08:00 |
|
|
Michael Goin
|
51b2333be1
|
[Perf] Optimize top-k search in apply_top_k_top_p_triton sampler (#37225)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-03-17 11:35:17 -07:00 |
|
|
Kunshang JiandHarry Mellor
|
16d2ad1d38
|
[Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-04 09:49:47 +00:00 |
|
|
Kunshang Ji
|
8ad54a991b
|
[Platform] Add current_platform.num_compute_units interface (#35042)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
|
2026-02-24 22:22:49 -08:00 |
|
|
Kunshang Ji
|
92510edc32
|
remove cuda check in top_k_top_p_triton kernel (#35011)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-02-24 22:22:31 -08:00 |
|
|
Xiao Li
|
30132cd144
|
Fix apply_top_k_top_p_triton called by non-cuda logits Tensor (#35030)
Signed-off-by: Xiao Li <ilx@meta.com>
|
2026-02-21 21:11:54 -08:00 |
|
|
|
c656ba3b4d
|
[Kernel] Triton-based Top-k and Top-p sampler kernels (#33538)
Signed-off-by: js_park <cakeng@naver.com>
Signed-off-by: Jongseok Park <37990712+cakeng@users.noreply.github.com>
Signed-off-by: Sunga Kim <sunga.kim@berkeley.edu>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Sunga Kim <sunga.kim@berkeley.edu>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-02-17 23:14:30 +00:00 |
|
|
Pleaplusone
|
77c16df31d
|
[ROCm][Bugfix] Disable hip sampler to fix deepseek's accuracy issue on ROCm (#32413)
Signed-off-by: ganyi <ygan@amd.com>
|
2026-01-15 16:35:47 +00:00 |
|
|
Nick Hill
|
10ef65eded
|
[BugFix] Fix bad words with speculative decoding (#31908)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-07 15:46:42 -05:00 |
|
|
Harry MellorandCyrus Leung
|
9eec282cb5
|
Guard FlashInfer sampler using the same check as FlashInfer attention backend (#29415)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-11-28 08:34:48 -08:00 |
|
|
Xiao Li
|
ed6ae1e36a
|
[AITER] [ROCm] Fix crash when loading llama4 model with old aiter version installed, fallback to forward_native implementation (#29124)
Signed-off-by: Xiao Li <ilx@meta.com>
|
2025-11-20 17:54:35 -08:00 |
|
|
vllmellm
|
0af3d4f0df
|
[FEAT] [AITER] [ROCm] integrate aiter sampling ops (#26084)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2025-11-18 17:28:34 +00:00 |
|
|
Zhang Xiangze
|
7bdb42b2f2
|
[CPU]Avoid repeated random sample compile (#28260)
Signed-off-by: Zhang Xiangze <Xiangze.Zhang@arm.com>
|
2025-11-07 11:03:57 +00:00 |
|
|
Nick Hill
|
c2ed069b32
|
[BugFix] Fix mixed penalties batch with async scheduling (#27910)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-11-01 10:51:24 -07:00 |
|
|
Wentao Ye
|
52efc34ebf
|
[Log] Optimize Startup Log (#26740)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2025-10-24 19:27:04 -04:00 |
|
|
Jonathan Chen
|
ca76486a16
|
[Chore] Separate out vllm.utils.platform_utils.py (#27374)
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
|
2025-10-23 19:08:06 +00:00 |
|
|
Isotr0py
|
6ac5e06f7c
|
[Chore] Clean up pytorch helper functions in vllm.utils (#26908)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
|
2025-10-18 09:48:22 -07:00 |
|
|
Jee Jee Li
|
fec2b341ad
|
[Kernel] Lazy import FlashInfer (#26977)
|
2025-10-17 04:48:18 +00:00 |
|
|
Akash kaothalkarandAkash Kaothalkar
|
f7d318de2b
|
[Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987)
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
|
2025-10-15 22:36:59 -07:00 |
|
|
Michael Goin
|
e66d787bce
|
Disable FlashInfer sampler by default (#26859)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-10-15 02:35:18 +00:00 |
|
|
ihb2032
|
4a61950f4d
|
[Hardware][CPU] Disable torch.compile for RISC-V to prevent APIError (#26693)
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn
|
2025-10-13 07:56:01 -07:00 |
|
|
Harry Mellor
|
8fcaaf6a16
|
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-12 09:51:31 -07:00 |
|
|
Harry Mellor
|
2f99f2f506
|
Tidy vllm/config/__init__.py to only add classes and functions (#26405)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-08 07:10:00 -07:00 |
|
|
|
6ebaf43ee4
|
[V1] Logit processors for rejection sampler (#19482)
Signed-off-by: southfreebird <yvorott@gmail.com>
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Signed-off-by: Sergei Skvortsov <yvorott@gmail.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
|
2025-10-07 13:02:49 -07:00 |
|