iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Cyrus Leung e6d4f3c254 [Bugfix] Fix pre-commit (#29601) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-27 02:23:06 -08:00
Morrison TurnanskyadabeytaLuka GovedičHarry Mellor
 		0838b52e2e [Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-27 01:55:58 -08:00
Woosuk Kwon da3222f371 [Model Runner V2] Implement multi-step Eagle with CUDA graph (#29559) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-27 00:09:41 -08:00
Woosuk Kwon 11ea5ec1ff [Model Runner V2] Refactor CudaGraphManager (#29583) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-26 21:37:59 -08:00
Woosuk Kwon ee80aee1ca [Model Runner V2] Minor cleanup for build_attn_metadata (#29576) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-26 20:10:12 -08:00
Woosuk Kwon 0aeb698b77 [Model Runner V2] Minor code cleanup (#29570) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-26 19:47:17 -08:00
Matthew Bonanni 77740191de [Attention][Async] Eliminate seq_lens_cpu in FlashAttention metadata building with DCP > 1 (#29449) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-26 18:48:43 -08:00
Johnny Yang ba1fcd84a7 [TPU] add tpu_inference (#27277) 
Signed-off-by: Johnny Yang <johnnyyang@google.com>
 2025-11-26 14:46:36 -08:00
Lucas Wilkinson 56539cddac [Core] Refactor padding logic and pad for CUDA graphs before attention metadata building (#28579) 2025-11-26 14:07:13 -05:00
Matthew Bonanni 430dd4d9eb [Attention] Remove imports from vllm/attention/__init__.py (#29342) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-26 10:53:15 -07:00
Pleaplusone d9d342d214 [Performance][MLA][ROCm] Remove redundant D2D copy in deepseek (#27457) 
Signed-off-by: ganyi <ygan@amd.com>
 2025-11-26 12:45:28 +08:00
Xieyang Xu 12866af748 dummy run corner case (#29433) 2025-11-26 00:20:35 +00:00
Andrey KhalyavinandTyler Michael Smith de75b0bb70 [BugFix] Fix initialization of draft model. (#29319) 
Signed-off-by: Andrey Khalyavin <halyavin@yandex-team.ru>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-11-25 18:45:58 -05:00
Nick Hill 4e57c6587f [Core] Support logprobs with spec decode + async scheduling (#29223) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-25 12:55:24 -08:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> a1f2676879 Scheduled removal of override_pooler_config and disable_log_requests (#29402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-25 16:08:57 +00:00
Yifan QiaoandChen Zhang 48ddb02b79 [Hybrid Allocator] Support KV cache groups with different block_size (#29143) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-11-25 10:30:57 -05:00
Eldar Kurtić 0231ce836a Revert back to torch.equal over torch.allclose from #28819 (#29086) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
 2025-11-25 14:23:38 +00:00
Roger Wang c2c661af9b [Bugfix] Fix overallocation in MM profiling (#29386) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-25 12:38:36 +00:00
Nicolò LucchesiandEkagra Ranjan 798e87db5c [Core] Generalize Encoder-Decoder seq_lens computation to avoid Whisper hardcoded logic (#29268) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-11-25 11:32:11 +00:00
Avishek Goswami 32c40b95e0 [BugFix] bad_words filtering ineffective when n > 1 (#29313) 
Signed-off-by: GOavi101 <1704178@kiit.ac.in>
 2025-11-25 09:36:34 +00:00
wang.yuqi 67fc16cd8c [Bugfix] If chunked_prefill is disabled, end the scheduling early. (#28911) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-11-25 16:06:09 +08:00
Rémi Delacourt 12c007e288 EAGLE Support DP>1 (#26086) 
Signed-off-by: Rémi Delacourt <remi@mistral.ai>
Signed-off-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Signed-off-by: remi <remi@mistral.ai>
 2025-11-25 07:32:21 +00:00
Jiangyun Zhu 81db702ed2 [Attention] add _cudagraph_support for linear attention (#28934) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-25 12:25:20 +08:00
Isotr0pysergeywangmanayangmanayangRoger Wang
 		92effb07a4 [Model] Add HunyuanOCR support (#29327) 
Signed-off-by: manayang <jackmanayang@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: sergeywang <sergeywang@tencent.com>
Co-authored-by: manayang <jackmanayang@gmail.com>
Co-authored-by: manayang <manayang@tencent.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-25 03:28:51 +00:00
Mark McLoughlin 9cf4edae6e [Metrics] Scheduled removal of deprecated metrics (#29330) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-25 11:15:13 +08:00
gbyu-amdandguanbao cb7214d8ea [ROCm][MLA] enable fp8 MLA decode on ROCm (#28032) 
Signed-off-by: guanbao <gyu@amd.com>
Signed-off-by: Guanbao Yu <gyu@amd.com>
Signed-off-by: gbyu-amd <Guanbao.Yu@amd.com>
Co-authored-by: guanbao <gyu@amd.com>
 2025-11-25 10:15:02 +08:00
Pleaplusone 77e10c9cab [Perf][Deepseek] optimize gather_and_maybe_dequant_cache kernel's perf for extremely long sequence (#28029) 
Signed-off-by: ganyi <ygan@amd.com>
 2025-11-24 19:05:46 -07:00
Hanjie QiuandBenjamin Chislett 5f9679a43b [Spec Decode] Add support for EAGLE3 heads that do not use_aux_hidden_states (#27688) 
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-11-24 20:13:12 -05:00
Chen Zhang 71df2a57ef [Hybrid Allocator] Better layer padding strategy for gpt-oss eagle (#29303) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-11-24 14:28:32 -08:00
Woosuk Kwon f32c7d6f54 [Model Runner V2] Simplify Eagle bookkeeping with num_rejected (#29347) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-24 13:54:59 -08:00
Woosuk Kwon 97588c4d12 [Model Runner V2] Add minor clarification comments for Eagle (#29332) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-24 11:28:56 -08:00
Woosuk Kwon cec418b5df [Model Runner V2] Change Numba AoT to JIT (#29328) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-24 09:34:37 -08:00
Woosuk Kwon cc313cb73d [Model Runner V2] Implement Single-step Eagle 1 (#29300) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-24 09:32:27 -08:00
WeiQing ChenandSunChenxiang123 2601f18a82 [EPLB] Optimize EPLB for Async Rearrange Experts (#22179) 
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: SunChenxiang123 <1291824390@qq.com>
 2025-11-24 09:08:29 -05:00
Didier DurandandHarry Mellor eca7a8fb59 [Doc]: fix typos in various files (#29230) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-24 11:10:48 +00:00
Roger Wang 0ff70821c9 [Core] Deprecate xformers (#29262) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-24 04:18:55 +00:00
tongqiu 5253f4276f [ROCm] Support for Whisper v1 with Aiter Unified Attention and Aiter Flash Attention (#28376) 
Signed-off-by: apinge <Tong.Qiu2@amd.com>
 2025-11-24 03:26:00 +00:00
Woosuk Kwon 3e1ad40655 [Model Runner V2] Add apply_temperature option to gumbel_sample (#29276) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-23 14:13:00 -08:00
Woosuk Kwon 62d54ba46d [Model Runner V2] Optimize CUDA graph capture time (#29275) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-23 11:15:32 -08:00
Woosuk Kwon b004c00418 [Model Runner V2] Support spec decoding [1/N] (#29274) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-23 10:09:06 -08:00
Woosuk Kwon 7f12c82fa6 [Model Runner V2] Change bookkeeping logic in preparation for spec decoding (#29194) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-23 09:42:52 -08:00
Woosuk Kwon 20ee418adc [Model Runner V2] Minor fix for cudagraph_utils (#29256) 2025-11-22 20:12:50 -08:00
Yizhou df78aeef08 Refactor: Move CUDA graph dispatch logic earlier (#27382) 
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
 2025-11-22 16:10:31 -05:00
Nick Hill 7df331c66b [BugFix] Fix chunked prompt logprobs + preemption (#29071) 2025-11-22 16:07:18 -05:00
Fadi Arafeh 730bd35378 [perf][cpu] Accelerate paged attention GEMMs (QK, PV) on Arm CPUs with NEON (#29193) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-11-22 09:04:36 -08:00
Nick Hill d44a63c6d6 [BugFix] Fix returned logprobs with spec decode + prefill chunking (#29216) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-22 22:41:25 +08:00
Nicolò Lucchesi 066209a045 [Attention] Refactor FA block_size limitations to hybrid models only (#29084) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-11-22 06:38:44 -08:00
Cyrus Leung 5a4802588e [Misc] Further clean up chunked prefill and prefix caching init (#29186) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-22 19:34:15 +08:00
Woosuk Kwon e9056056fb [Model Runner V2] Limit cudagraph size to max decode batch size (#29221) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-21 20:21:35 -08:00
Jie Luo 5c8f2adf50 [Bugfix] Fix block size in block_table with PCP (#29094) 
Signed-off-by: Livinfly <luojie3m@gmail.com>
 2025-11-22 01:34:28 +00:00