iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Haco d811b442d3 [Bugfix] DeepSeek V3.2 MTP metadata & CUDA graph issues (#26779) 
Signed-off-by: xiaohajiayou <923390377@qq.com>
 2025-11-01 10:52:43 -04:00
wangxiyuanandHarry Mellor 30a14b034f [V0 deprecation] Remove VLLM_USE_V1 usage in platform and v1 module (#27798) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-01 10:17:45 +00:00
Nick Hill 0cdbe7b744 [Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
Chen Zhang df334868ca [Hybrid] A simpler algorithm to find kernel_block_size (#26476) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-31 21:30:28 +00:00
Matthew Bonanni f29aeb5a25 Add FLASHINFER_MLA to test_mla_backends and add B200 CI run (#27663) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-31 11:12:19 -07:00
GuanLuoandNicolò Lucchesi d6517be3cd [Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-31 10:16:00 -07:00
Madeesh Kannan 675704ac01 [Bugfix] Allow 64-bit integer values for LoRA IDs to avoid overflow/truncation (#27876) 
Signed-off-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2025-10-31 16:58:42 +00:00
Huamin Li 933cdea440 [BugFix] Don’t compute reorder threshold when there are no attention groups (#27861) 2025-10-31 11:36:18 +00:00
Isotr0py 3933f18a5e [Bugfix] Avoid too small block m/n for FlexAttention kernel option (#27853) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-31 19:33:12 +08:00
Nick Hill c9791f1813 [BugFix] Fix broken import in initialize_ray_cluster() (#27838) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-30 16:26:13 -07:00
Jialin Ouyang 4b68c4a55b [Core][Perf] Only invoke save_new_computed_blocks when computed blocks are not empty (#27799) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-30 19:47:30 +00:00
Sumanth R Hegde 4917002523 [Fix] Skip record_sleep_state logic in PrometheusStatsLogger if not in dev mode (#27789) 
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
 2025-10-30 19:26:27 +00:00
Jialin Ouyang 4574d48bab [Core][Bookkeeping] Update cu_num_accepted_tokens for all req_index (#27629) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-30 11:52:36 -07:00
Tyler Michael SmithandNick Hill ab98f6556f [Bugfix] Fix 2 precommit issues - (mamba_block_size, kv_cache_config) (#27811) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-30 11:52:18 -07:00
Ilya MarkovandTyler Michael Smith 60f76baa66 [Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for device selection on cuda-like devices (#27564) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-30 11:41:44 -04:00
Zhewen Li 0fe0140408 [KV offload] Enable CPU KV offload on CUDA alike Platforms (#27770) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-10-30 22:10:29 +08:00
Zhiyuan Li 4e68cc9b6a [Model] Introduce Kimi Linear to vLLM (#27809) 
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-30 21:02:27 +08:00
Sairam Pillai 74374386e2 [Bugfix] Improve GPU validation logging in Ray fallback scenarios (#25775) 
Signed-off-by: Sairam Pillai <sairam.pillai61@gmail.com>
 2025-10-30 11:57:59 +00:00
Wentao Ye c01f6e525f [CI] Fix mypy for vllm/v1/core and vllm/v1/engine (#27108) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-30 11:32:17 +00:00
Lucas Wilkinson b5d70751d8 [BugFix] Reordering extend logic fix (#27739) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-29 21:39:34 -07:00
Nick Hill 2ce5c5d3d6 [BugFix] Handle unscheduled requests properly when async scheduling (#27756) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-29 21:04:25 -07:00
Wentao Ye b5d90f7400 [Bug] Fix DBO IMA issue for DeepEPHT (#27666) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-29 16:28:27 -04:00
Nicolò LucchesiandMark McLoughlin accb8fab07 [KVConnector] Add metrics to Prometheus-Grafana dashboard (#26811) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-29 18:44:49 +00:00
Braulio Dumba 1da3309ace [Core] Exposing engine sleep & wake_up state as prometheus metrics (#24176) 
Signed-off-by: Braulio Dumba <Braulio.Dumba@ibm.com>
 2025-10-29 09:32:01 -07:00
Jialin Ouyang 0b51c9bd8b [Core] Early return in SlidingWindowManager.remove_skipped_blocks (#27673) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-29 01:32:33 +00:00
Nick Hill 4fe5895361 [AsyncScheduling] Make async overlap work with logprobs (#27615) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-28 22:35:54 +00:00
Or Ozeri 111faf1118 [Core] Scheduler: Publish connector events after output (#25875) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-10-28 21:01:33 +00:00
Lucas Wilkinson 141e6a0505 [Misc] Make reorder batch also separate extends (#27367) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-28 10:55:10 -07:00
Cyrus Leung f5710ef02a [Misc] Make LayerBlockType a Literal instead of Enum (#27658) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-28 16:23:35 +00:00
Mohammad Miadh AngkadandYe Qi a8c02fb5bf [Bugfix][CI] Fix v1 attention backend tests and add CI coverage (#26597) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-28 11:42:05 -04:00
22quinnandZhuohan Li 2abbd351ef [Core] Enable async scheduling for external_launcher mode (#27394) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-10-28 13:52:47 +00:00
Cyrus Leung f58d9b6404 [Misc] Separate out utils.counter and move utils.Device to engine (#27588) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-28 12:20:46 +00:00
Jialin Ouyang b46e4a06f1 [Core][Bookkeeping Optimization] Update against numpy view of is_token_ids tensor (#27618) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-28 08:13:10 +00:00
Chauncey 61fbfe5274 [Bugfix] fixed inconsistent finish_reason handling between V0 and V1 engines (#27555) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-28 02:18:08 +00:00
usberkeley 69f064062b Code quality improvements: version update, type annotation enhancement, and enum usage simplification (#27581) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
 2025-10-27 17:50:22 +00:00
Cyrus Leung 6ebffafbb6 [Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Yeshwanth N 71b1c8b667 [Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
Cyrus Leung 4c5f632165 [Misc] Simplify max tokens in multimodal registry (#27500) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-24 23:56:01 -07:00
Kuntai Du b853540388 [Core][Hybrid allocator + kv connector 1/n] Enable hybrid allocator + KV cache connector (#25712) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
 2025-10-24 23:34:18 -07:00
Zhuohan Li 56ed7609a9 Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502) 2025-10-25 05:31:43 +00:00
Wentao Ye 52efc34ebf [Log] Optimize Startup Log (#26740) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-24 19:27:04 -04:00
Lehua Ding 0402428200 [Perf][Async Scheduling] Remove CPU->GPU sync in dummy_run (#27455) 
Signed-off-by: Lehua Ding <lehuading@tencent.com>
 2025-10-24 20:45:36 +00:00
Ming Yang 0f67d4d962 [Attention] Add MLA prefill backend: trtllm_ragged_attention_deepseek (#26397) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-10-24 10:24:08 -07:00
fhl2000Luka GovedičHarry Mellor
 		284cc92275 [MISC] cudagraph_capture_sizes related improvements (#26016) 
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-24 05:11:05 -07:00
Aaron Pham d4c574c39f [Chore] remove structural tags logging lines (#27451) 2025-10-24 05:35:45 +00:00
fhl2000 85fee74b33 [Bugfix][CI] Move resolving cudagraph_mode before initializing attn_metadata_builder (#27427) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
 2025-10-23 20:31:14 -07:00
KebeandRui Qiao 51dd14ac2b [Bugfix][DP] Fix creating too many DP Placement Groups (#26880) 
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-23 20:16:51 +00:00
Matthew Bonanni dbfbf9f324 [Attention] Fix FlashMLA metadata builder arguments for q_len > 1 (#27368) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-23 15:58:15 -04:00
Jonathan Chen ca76486a16 [Chore] Separate out vllm.utils.platform_utils.py (#27374) 
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
 2025-10-23 19:08:06 +00:00
Ilya MarkovandTyler Michael Smith 237cf6d32a [Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow startup time &c) (#26709) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-23 20:58:39 +08:00