 Matthew Bonanni
|
98f60e5acb
|
[6/N][Attention] Move utils to more appropriate locations (#32215)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-01-13 05:38:52 -08:00 |
|
|
Woosuk Kwon
|
0a7dd23754
|
[Model Runner V2] Add support for M-RoPE (#32143)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-12 13:37:43 -08:00 |
|
|
Woosuk Kwon
|
dec28688c5
|
[Model Runner V2] Minor refactor for logit_bias (#32209)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-12 13:08:30 -08:00 |
|
|
Woosuk Kwon
|
ca81811bfe
|
[Model Runner V2] Support logit_bias, allowed_token_ids, min_tokens (#32163)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-12 11:31:10 -08:00 |
|
|
Matthew Bonanni
|
20228cb851
|
[3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-01-12 09:13:56 -08:00 |
|
|
|
49e6b86c91
|
[Feature] Support recording expert indices for rollout router replay (#28284)
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: Hongxin Xu <70438206+xhx1022@users.noreply.github.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
|
2026-01-12 06:23:04 -08:00 |
|
|
Woosuk Kwon
|
025a32f9ed
|
[Model Runner V2] Remove async barrier (#32083)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-11 20:24:30 -08:00 |
|
|
Woosuk Kwon
|
19504ac07f
|
[Model Runner V2] Skip building deprecated fields in attn metadata (#32132)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-11 14:31:04 -08:00 |
|
|
Or Ozeri
|
4c16ba617f
|
[KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-01-11 08:05:36 +00:00 |
|
|
zhrrr
|
97ba96fbe9
|
[perf][async] support non cpu sync get logprob tensors for spec (#31336)
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-01-09 21:24:51 +00:00 |
|
|
Matthew Bonanni
|
2612ba9285
|
[1/N][Attention] Restructure attention: move files (#31916)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-01-09 13:10:24 -08:00 |
|
|
Michael Goin
|
d5ec6c056f
|
[UX] Add vLLM model inspection view (#29450)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-01-09 10:12:35 -07:00 |
|
|
Nick Hill
|
29ce48221c
|
[Cleanup] Remove obsolete spec decoding compatibility logic (#32003)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-09 05:44:18 +00:00 |
|
|
zhrrrandNick Hill
|
8ff4a99566
|
[Async][Feat] support apply penalty or bad_words for async + spec (#30495)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-09 02:31:50 +00:00 |
|
|
Max HuandMax Hu
|
6ebe34d6fa
|
[Feature] Add iteration level logging and enhance nvtx marker (#31193)
Signed-off-by: Max Hu <maxhu@nvidia.com>
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
|
2026-01-09 00:13:39 +00:00 |
|
|
Lucas WilkinsonandLuka Govedič
|
6cdf015c3c
|
[Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
|
2026-01-08 15:20:49 -08:00 |
|
|
Woosuk Kwon
|
7508243249
|
[Model Runner V2] Simplify BlockTables with UVA (#31965)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-08 10:24:26 -08:00 |
|
|
Nick Hill
|
a3d909ad2b
|
[Misc] Tidy up some spec decode logic in GPUModelRunner (#31591)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-08 09:10:07 -08:00 |
|
|
Nick Hill
|
287b37cda4
|
[BugFix] Fix spec decoding edge case bugs (#31944)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-08 15:31:03 +08:00 |
|
|
Ning Xie
|
c907d22158
|
[refactor] refactor memory constants usage (#31865)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2026-01-07 18:37:31 +00:00 |
|
|
Cyrus Leung
|
b7036c87a1
|
[Refactor] Clean up pooler modules (#31897)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-01-08 00:07:43 +08:00 |
|
|
Cyrus Leung
|
b665bbc2d4
|
[Chore] Migrate V0 attention utils (#31891)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-01-07 13:44:36 +00:00 |
|
|
weiyu
|
e7596371a4
|
[Refactor][TPU] Remove torch_xla path and use tpu-inference (#30808)
Signed-off-by: Wei-Yu Lin <weiyulin@google.com>
Signed-off-by: weiyu <62784299+weiyu0824@users.noreply.github.com>
|
2026-01-07 16:07:16 +08:00 |
|
|
tianshu-Michael-yu
|
efeaac92f2
|
[Bugfix] Fix race condition in async-scheduling for vlm model (#31841)
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
|
2026-01-07 06:45:10 +00:00 |
|
|
Cyrus Leung
|
aafd4d2354
|
[Chore] Try remove init_cached_hf_modules (#31786)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-01-07 12:34:04 +08:00 |
|
|
Benjamin ChislettandNick Hill
|
f7008ce1c4
|
[Perf] Async Scheduling + Speculative Decoding + Structured Outputs (#29821)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-06 18:50:37 +00:00 |
|
|
Ning Xie
|
6f5e653383
|
[Log] add log about gpu worker init snapshot and requested memory (#29493)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2026-01-06 17:32:55 +00:00 |
|
|
Wentao Ye
|
af9a7ec255
|
[Bug] Revert torch warning fix (#31585)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-01-05 22:31:21 +00:00 |
|
|
Nick Hill
|
43e3f8e4a9
|
[Misc] Various code simplifications (#31666)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-04 18:35:56 -08:00 |
|
|
wangxiyuan
|
bb4337b34c
|
[Platform] Deprecate seed_everything (#31659)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2026-01-04 18:34:04 -08:00 |
|
|
Nick Hill
|
da436f868a
|
[Minor] Small pooler output processing optimization (#31667)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-04 18:33:12 -08:00 |
|
|
Nick Hill
|
bd877162eb
|
[BugFix] Support online dense model DP without overhead (#30739)
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-02 23:36:38 +08:00 |
|
|
Nick Hill
|
6c2cfb62ff
|
[BugFix] Fix async scheduling for pooling models (#31584)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-31 14:48:51 -08:00 |
|
|
maang-h
|
b4bb5f312f
|
[Core] Remove unused num_tokens parameter from _init_model_kwargs (#31517)
Signed-off-by: maang <maang_h@163.com>
|
2025-12-30 20:47:23 -08:00 |
|
|
vintipandey
|
04147dcfa7
|
[Bugfix]Fix pooling model always disabled due to incorrect PP rank check (#31505)
Signed-off-by: vintipandey <vinti.pandey@gmail.com>
|
2025-12-30 11:27:10 -08:00 |
|
|
Nick Hill
|
3b312fb792
|
[Minor] Various small code cleanups/simplifications (#31508)
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-29 22:42:06 -08:00 |
|
|
Nick Hill
|
094fcce250
|
[BugFix] Re-fix async multimodal cpu tensor race condition (#31373)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: njhill <nickhill123@gmail.com>
|
2025-12-28 03:05:08 -08:00 |
|
|
Yifan QiaoandKuntaiDu
|
52bf066516
|
[Core][Hybrid allocator + connector] Support hybrid allocator + kv cache connector (#30166)
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Co-authored-by: KuntaiDu <kuntai@uchicago.edu>
|
2025-12-26 18:25:46 -08:00 |
|
|
|
ce1eafd1a5
|
[Core] Initialize LoRA support for tower and connector in multi-modal models (#26674)
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: Anexdeus <5142168@mail.ru>
|
2025-12-26 04:48:20 -08:00 |
|
|
Cyrus Leung
|
09dc7c690c
|
[Chore][1/2] Drop v0.14 deprecations (#31285)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-24 09:54:01 -08:00 |
|
|
Cyrus Leung
|
aa3868ecfe
|
[Chore] Remove unused noqas (#31263)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-24 05:38:46 -08:00 |
|
|
Michael Goin
|
8ee90c83f8
|
Add --max-model-len auto to auto-fit context to available memory (#29431)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2025-12-23 21:37:14 -08:00 |
|
|
Cyrus Leung
|
ca6a95ba25
|
[Chore] Simplify logic of _execute_mm_encoder (#31222)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-23 18:15:16 -08:00 |
|
|
Patrick von Platen
|
3faa8bee57
|
adapt voxtral (#31095)
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
|
2025-12-23 05:31:55 -08:00 |
|
|
Lucas Wilkinson
|
de71747655
|
[SpecDecode] Simplified alternative padded-speculation acceptance rate fix (#29845)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-22 13:06:10 -08:00 |
|
|
Boyuan Feng
|
8dd0db687b
|
[UX] improve profiler error message (#31125)
Signed-off-by: Boyuan Feng <boyuan@meta.com>
|
2025-12-22 08:45:59 -08:00 |
|
|
dengyunyang
|
8f8f469b1b
|
[BugFix] skip language model in Encoder (#30242)
Signed-off-by: dengyunyang <584797741@qq.com>
|
2025-12-22 05:25:59 -08:00 |
|
|
Lucas Wilkinson
|
5f6477d1d0
|
[BugFix] Fix TypeError: unhashable type: 'dict' when serving deepseek32 (#30924)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-19 16:07:54 -05:00 |
|
|
Nick Hill
|
b0b77c4655
|
[BugFix] Fix spec decode + structured outputs + preemption edge case (#30916)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-18 12:59:55 -08:00 |
|
|
Nick Hill
|
686cbaac64
|
[Cleanup] Remove unused ModelRunner V1 InputBatch.num_tokens field (#30218)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-18 09:17:00 -08:00 |
|