Yong Hoon Shin
|
69520bc695
|
Add logging for cudagraph related info (#29825)
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
|
2025-12-03 01:01:48 -08:00 |
|
 Arpit KhandelwalandLuka Govedič
|
d7284a2604
|
[Core] Rename PassConfig flags as per RFC #27995 (#29646)
Signed-off-by: arpitkh101 <arpit5khandelwal@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
|
2025-12-03 03:38:55 +00:00 |
|
Lucas Wilkinson
|
5cdd664509
|
[BugFix] Fix assert in build_for_cudagraph_capture (#29893)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-12-02 16:56:54 -08:00 |
|
Wushi Dong
|
0037b5746a
|
[Core] Eliminate redundant is_encoder_decoder lookups (20-40us/step) (#29800)
Signed-off-by: Wushi Dong <dongws@meta.com>
|
2025-12-02 07:08:07 +00:00 |
|
 Zhuohan LiandNick Hill
|
d0cd728907
|
[Core] Support reseting all running requests' KV while calling reset_prefix_cache (#28827)
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
|
2025-12-02 02:25:05 +00:00 |
|
Yifei Zhang
|
1ab8fc8197
|
Make PyTorch profiler gzip and CUDA time dump configurable (#29568)
Signed-off-by: Yifei Zhang <yifei.zhang1992@outlook.com>
|
2025-12-01 04:30:46 +00:00 |
|
Woosuk Kwon
|
ec38a7368d
|
[Model Runner V2] Use packed mask for prompt bin counts (#29756)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-30 14:15:42 -08:00 |
|
Cyrus Leung
|
64bc09ba27
|
[Core] Enable inputs_embeds_size separate from hidden_size (#29741)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-11-30 17:31:12 +08:00 |
|
 VensenandTJian
|
66b5840287
|
[Bugfix][sleepmode][fp8 kv cache]: Fix FP8 KV cache + sleep(level=2) gibberish output (#28783)
Signed-off-by: vensen <vensenmu@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2025-11-30 14:24:25 +08:00 |
|
Woosuk Kwon
|
f223ed4181
|
[Model Runner V2] Fuse penalties and temperature into single kernel (#29720)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-29 02:29:16 -08:00 |
|
Woosuk Kwon
|
6afc0ffaf6
|
[Model Runner V2] Add sample/ directory and reorganize files (#29719)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-29 00:41:01 -08:00 |
|
 Jee Jee LiandCyrus Leung
|
39e63dec7c
|
[LoRA] Cleanup LoRA unused code (#29611)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-11-28 22:52:58 -08:00 |
|
Woosuk Kwon
|
4a80ad0a25
|
[Model Runner V2] Don't use UVA buffer for prefill_len (#29713)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-28 20:27:16 -08:00 |
|
Lucas Wilkinson
|
e23f665d83
|
[BugFix] Fix DBO failing with TypeError: 'NoneType' object is not iterable (#29698)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-11-28 20:19:01 -08:00 |
|
Woosuk Kwon
|
ca1b1e7296
|
[Model Runner V2] Refactor prefill token preparation (#29712)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-28 19:49:17 -08:00 |
|
Woosuk Kwon
|
1dcafb3dea
|
[Model Runner V2] Support penalties using bin counts (#29703)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-28 17:53:17 -08:00 |
|
Benjamin Chislett
|
1986de1375
|
[Perf] Optimize EAGLE prepare_inputs_padded with triton kernels (#28597)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2025-11-28 22:25:05 +00:00 |
|
Didier Durand
|
fae6943068
|
[Doc]: fixing typos in multiple files. (#29685)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
|
2025-11-28 08:41:41 -08:00 |
|
Lucas Wilkinson
|
be493e0b3c
|
[BugFix] Fix new nightly failures (#29578)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-11-27 13:45:38 -08:00 |
|
Woosuk Kwon
|
ae0ce1be27
|
[Model Runner V2][BugFix] Keep reference to GPU tensors in AsyncOutput (#29623)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-27 12:38:53 -08:00 |
|
Matthew Bonanni
|
fc1d8be3dc
|
[Attention] Update attention imports (#29540)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-11-27 11:19:09 -05:00 |
|
Didier Durand
|
66d3d5422c
|
[Doc]: fixing typos in diverse files (#29492)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
|
2025-11-27 07:15:50 -08:00 |
|
Cyrus Leung
|
e6d4f3c254
|
[Bugfix] Fix pre-commit (#29601)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-11-27 02:23:06 -08:00 |
|
  
|
0838b52e2e
|
[Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847)
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-11-27 01:55:58 -08:00 |
|
Woosuk Kwon
|
da3222f371
|
[Model Runner V2] Implement multi-step Eagle with CUDA graph (#29559)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-27 00:09:41 -08:00 |
|
Woosuk Kwon
|
11ea5ec1ff
|
[Model Runner V2] Refactor CudaGraphManager (#29583)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-26 21:37:59 -08:00 |
|
Woosuk Kwon
|
ee80aee1ca
|
[Model Runner V2] Minor cleanup for build_attn_metadata (#29576)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-26 20:10:12 -08:00 |
|
Woosuk Kwon
|
0aeb698b77
|
[Model Runner V2] Minor code cleanup (#29570)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-26 19:47:17 -08:00 |
|
Johnny Yang
|
ba1fcd84a7
|
[TPU] add tpu_inference (#27277)
Signed-off-by: Johnny Yang <johnnyyang@google.com>
|
2025-11-26 14:46:36 -08:00 |
|
Lucas Wilkinson
|
56539cddac
|
[Core] Refactor padding logic and pad for CUDA graphs before attention metadata building (#28579)
|
2025-11-26 14:07:13 -05:00 |
|
Matthew Bonanni
|
430dd4d9eb
|
[Attention] Remove imports from vllm/attention/__init__.py (#29342)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-11-26 10:53:15 -07:00 |
|
Xieyang Xu
|
12866af748
|
dummy run corner case (#29433)
|
2025-11-26 00:20:35 +00:00 |
|
 Andrey KhalyavinandTyler Michael Smith
|
de75b0bb70
|
[BugFix] Fix initialization of draft model. (#29319)
Signed-off-by: Andrey Khalyavin <halyavin@yandex-team.ru>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2025-11-25 18:45:58 -05:00 |
|
Nick Hill
|
4e57c6587f
|
[Core] Support logprobs with spec decode + async scheduling (#29223)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-11-25 12:55:24 -08:00 |
|
Roger Wang
|
c2c661af9b
|
[Bugfix] Fix overallocation in MM profiling (#29386)
Signed-off-by: Roger Wang <hey@rogerw.io>
|
2025-11-25 12:38:36 +00:00 |
|
 Nicolò LucchesiandEkagra Ranjan
|
798e87db5c
|
[Core] Generalize Encoder-Decoder seq_lens computation to avoid Whisper hardcoded logic (#29268)
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2025-11-25 11:32:11 +00:00 |
|
Rémi Delacourt
|
12c007e288
|
EAGLE Support DP>1 (#26086)
Signed-off-by: Rémi Delacourt <remi@mistral.ai>
Signed-off-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Signed-off-by: remi <remi@mistral.ai>
|
2025-11-25 07:32:21 +00:00 |
|
   
|
92effb07a4
|
[Model] Add HunyuanOCR support (#29327)
Signed-off-by: manayang <jackmanayang@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: sergeywang <sergeywang@tencent.com>
Co-authored-by: manayang <jackmanayang@gmail.com>
Co-authored-by: manayang <manayang@tencent.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2025-11-25 03:28:51 +00:00 |
|
 Hanjie QiuandBenjamin Chislett
|
5f9679a43b
|
[Spec Decode] Add support for EAGLE3 heads that do not use_aux_hidden_states (#27688)
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2025-11-24 20:13:12 -05:00 |
|
Woosuk Kwon
|
f32c7d6f54
|
[Model Runner V2] Simplify Eagle bookkeeping with num_rejected (#29347)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-24 13:54:59 -08:00 |
|
Woosuk Kwon
|
97588c4d12
|
[Model Runner V2] Add minor clarification comments for Eagle (#29332)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-24 11:28:56 -08:00 |
|
Woosuk Kwon
|
cec418b5df
|
[Model Runner V2] Change Numba AoT to JIT (#29328)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-24 09:34:37 -08:00 |
|
Woosuk Kwon
|
cc313cb73d
|
[Model Runner V2] Implement Single-step Eagle 1 (#29300)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-24 09:32:27 -08:00 |
|
 WeiQing ChenandSunChenxiang123
|
2601f18a82
|
[EPLB] Optimize EPLB for Async Rearrange Experts (#22179)
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: SunChenxiang123 <1291824390@qq.com>
|
2025-11-24 09:08:29 -05:00 |
|
 Didier DurandandHarry Mellor
|
eca7a8fb59
|
[Doc]: fix typos in various files (#29230)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-11-24 11:10:48 +00:00 |
|
Woosuk Kwon
|
3e1ad40655
|
[Model Runner V2] Add apply_temperature option to gumbel_sample (#29276)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-23 14:13:00 -08:00 |
|
Woosuk Kwon
|
62d54ba46d
|
[Model Runner V2] Optimize CUDA graph capture time (#29275)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-23 11:15:32 -08:00 |
|
Woosuk Kwon
|
b004c00418
|
[Model Runner V2] Support spec decoding [1/N] (#29274)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-23 10:09:06 -08:00 |
|
Woosuk Kwon
|
7f12c82fa6
|
[Model Runner V2] Change bookkeeping logic in preparation for spec decoding (#29194)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-23 09:42:52 -08:00 |
|
Woosuk Kwon
|
20ee418adc
|
[Model Runner V2] Minor fix for cudagraph_utils (#29256)
|
2025-11-22 20:12:50 -08:00 |
|