iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Roger Wang 16abe6b85a [Misc] Set default torch num threads for input processing (#31879) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-12 10:28:16 -08:00
Matthew Bonanni 20228cb851 [3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-12 09:13:56 -08:00
Asaf Joseph Gardin 8fb2c135be [Bugfix] Fix stale SSM state for new Mamba requests scheduled as decode (#32118) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-01-12 17:02:38 +00:00
Or Ozeri 9cddbdba6d OffloadingConnector: Add cpu_bytes_to_use configuration (#24498) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-12 15:00:43 +00:00
Hongxin Xu22quinnarlenxu
 		49e6b86c91 [Feature] Support recording expert indices for rollout router replay (#28284) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: Hongxin Xu <70438206+xhx1022@users.noreply.github.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
 2026-01-12 06:23:04 -08:00
Isotr0py 9dbe1fe960 [Bugfix] Fix missing scale passing for encoder Triton Attention implementation (#32149) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-12 11:13:41 +00:00
Woosuk Kwon 025a32f9ed [Model Runner V2] Remove async barrier (#32083) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-11 20:24:30 -08:00
Woosuk Kwon 19504ac07f [Model Runner V2] Skip building deprecated fields in attn metadata (#32132) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-11 14:31:04 -08:00
rongfu.leng d70249e2e9 [Misc] fix this log format not space (#32112) 
Signed-off-by: lengrongfu <lenronfu@gmail.com>
 2026-01-11 05:01:16 -08:00
Or Ozeri 4c16ba617f [KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-11 08:05:36 +00:00
Or Ozeri 2a4dbe24ea [BugFix] Wait for compute before offloading KV to CPU (#31341) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-10 22:25:08 +00:00
Vadim Gimpelson e15a5ff07b [MISC] Add strict contiguity check for FlashInfer attention tensors (#32008) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-01-10 12:40:05 -08:00
Or Ozeri 028599739d [BugFix] scheduler: Fix resuming of preempted requests after async load (#31583) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-10 12:39:25 -08:00
jvlunteren b8bf5c45bb [Kernel] Optimize Sliding Window Attention in 3D Triton Kernel (#31984) 
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
 2026-01-10 18:13:44 +00:00
FrelamandRoger Wang a01a1c0d69 [Bugfix] fix encoder cache leak of waiting requests in scheduler to solve stuck in CPU scheduling (#31857) 
Signed-off-by: frelam <frelam112233@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-10 06:27:58 +00:00
Lucas Wilkinson da6709c9fe [Misc] Delay deprecation of CommonAttentionMetadata properties (#32074) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 21:06:44 -08:00
Lucas Kabela ea6d067a2a [Misc][LLaMa4] Compile LLaMa Vision Encoder (#30709) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-09 22:01:38 -05:00
Wentao Ye e18464a57d [Perf] Optimize async scheduling placeholder using empty (#32056) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-10 00:46:11 +00:00
Matthew Bonanni 0308901975 [2/N][Attention] Fix pre-commit errors (#32052) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-10 00:27:15 +00:00
zhrrr 97ba96fbe9 [perf][async] support non cpu sync get logprob tensors for spec (#31336) 
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2026-01-09 21:24:51 +00:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Wentao Ye 28ae32a5d3 [Refactor] Remove numpy split in async scheduling (#32034) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-09 19:09:02 +00:00
Yifan Qiao cd4a95e3aa [Feat][Core] Support multiple KV cache groups in Hybrid KV Coordinator (#31707) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
 2026-01-09 10:53:20 -08:00
Michael Goin d5ec6c056f [UX] Add vLLM model inspection view (#29450) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-09 10:12:35 -07:00
R3hankhan 8e27663b6a [CPU] Add head sizes 80 and 112 with vec16 fallback (#31968) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-01-09 22:14:46 +08:00
Andreas Karatzas e02706d2d2 [ROCm][CI][V1] Fix nixl_connector test failure and achieve CUDA parity in test_async_scheduling (#32000) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-09 20:48:32 +08:00
vllmellm 1a19e9cd87 [Bugfix][ROCm]Fix Qwen3-Next-80B-A3B-Thinking inference and optimize non-standard block size (544) support under rocm_atten (#31380) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-09 19:28:02 +08:00
Cyrus Leung c8ed39b9dd [Model] Reorganize pooling layers (#31973) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-09 11:02:14 +00:00
Nick Hill 29ce48221c [Cleanup] Remove obsolete spec decoding compatibility logic (#32003) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 05:44:18 +00:00
zhrrrandNick Hill 8ff4a99566 [Async][Feat] support apply penalty or bad_words for async + spec (#30495) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 02:31:50 +00:00
Max HuandMax Hu 6ebe34d6fa [Feature] Add iteration level logging and enhance nvtx marker (#31193) 
Signed-off-by: Max Hu <maxhu@nvidia.com>
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
 2026-01-09 00:13:39 +00:00
Nick Hill 11cec296dd [BugFix] Add spec-decode-incompatible request param validation (#31982) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 00:08:21 +00:00
Lucas WilkinsonandLuka Govedič 6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
Woosuk Kwon 7508243249 [Model Runner V2] Simplify BlockTables with UVA (#31965) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-08 10:24:26 -08:00
Nick Hill a3d909ad2b [Misc] Tidy up some spec decode logic in GPUModelRunner (#31591) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-08 09:10:07 -08:00
Cyrus Leung d1b6fe007f [Chore] Further cleanup pooler (#31951) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 02:16:21 -08:00
Lumosis b634e619bb Decouple page_size_bytes calculation in AttentionSpec for TPU/RPA Compatibility. (#31635) 
Signed-off-by: Lihao Ran <imlihao.ran@gmail.com>
Signed-off-by: Lumosis <30372757+Lumosis@users.noreply.github.com>
 2026-01-08 09:00:07 +00:00
Rabi Mishra 107cf8e92f fix(rocm): Add get_supported_kernel_block_sizes() to ROCM_ATTN (#31712) 
Signed-off-by: rabi <ramishra@redhat.com>
 2026-01-08 15:46:07 +08:00
prashanth058 d3235cb503 [Fix] Enable mm_processor_cache with vision LoRA (#31927) 
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
 2026-01-08 15:31:51 +08:00
Nick Hill 287b37cda4 [BugFix] Fix spec decoding edge case bugs (#31944) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-08 15:31:03 +08:00
Nick Hill 10ef65eded [BugFix] Fix bad words with speculative decoding (#31908) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-07 15:46:42 -05:00
Ning Xie c907d22158 [refactor] refactor memory constants usage (#31865) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-07 18:37:31 +00:00
Marko Rosenmueller 1d9e9ae8a4 [Bugfix]: prevent leaking tokens in crash log (#30751) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2026-01-07 16:15:19 +00:00
Cyrus Leung b7036c87a1 [Refactor] Clean up pooler modules (#31897) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 00:07:43 +08:00
R3hankhan 1ab055efe6 [OpenAI] Extend VLLMValidationError to additional validation parameters (#31870) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-01-07 14:45:49 +00:00
Cyrus Leung b665bbc2d4 [Chore] Migrate V0 attention utils (#31891) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-07 13:44:36 +00:00
vllmellm 41cfa50632 [ROCm][AITER] fix wrong argument passed to AITER flash_attn_varlen_func (#31880) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-07 11:25:03 +00:00
weiyu e7596371a4 [Refactor][TPU] Remove torch_xla path and use tpu-inference (#30808) 
Signed-off-by: Wei-Yu Lin <weiyulin@google.com>
Signed-off-by: weiyu <62784299+weiyu0824@users.noreply.github.com>
 2026-01-07 16:07:16 +08:00
tianshu-Michael-yu efeaac92f2 [Bugfix] Fix race condition in async-scheduling for vlm model (#31841) 
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
 2026-01-07 06:45:10 +00:00
Lucas Wilkinson c7a79d41a0 [Attention][3/n] Remove usage of deprecated seq_lens_cpu and num_computed_tokens_cpu CommonAttentionMetadata properties (#31850) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-07 13:31:34 +08:00