iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
10,937 Commits 3 Branches 0 Tags
2c19d96777939dd3473eabfacbe1948a3ea0b4be
Commit Graph
390 Commits
Author SHA1 Message Date
wangxiyuanandHarry Mellor 30a14b034f [V0 deprecation] Remove VLLM_USE_V1 usage in platform and v1 module (#27798) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-01 10:17:45 +00:00
Nick Hill 0cdbe7b744 [Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
GuanLuoandNicolò Lucchesi d6517be3cd [Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-31 10:16:00 -07:00
Ilya MarkovandTyler Michael Smith 60f76baa66 [Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for device selection on cuda-like devices (#27564) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-30 11:41:44 -04:00
Wentao Ye c01f6e525f [CI] Fix mypy for vllm/v1/core and vllm/v1/engine (#27108) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-30 11:32:17 +00:00
Braulio Dumba 1da3309ace [Core] Exposing engine sleep & wake_up state as prometheus metrics (#24176) 
Signed-off-by: Braulio Dumba <Braulio.Dumba@ibm.com>
 2025-10-29 09:32:01 -07:00
22quinnandZhuohan Li 2abbd351ef [Core] Enable async scheduling for external_launcher mode (#27394) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-10-28 13:52:47 +00:00
Cyrus Leung f58d9b6404 [Misc] Separate out utils.counter and move utils.Device to engine (#27588) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-28 12:20:46 +00:00
Cyrus Leung 6ebffafbb6 [Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Yeshwanth N 71b1c8b667 [Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
Zhuohan Li 56ed7609a9 Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502) 2025-10-25 05:31:43 +00:00
Wentao Ye 52efc34ebf [Log] Optimize Startup Log (#26740) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-24 19:27:04 -04:00
KebeandRui Qiao 51dd14ac2b [Bugfix][DP] Fix creating too many DP Placement Groups (#26880) 
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-23 20:16:51 +00:00
Ilya MarkovandTyler Michael Smith 237cf6d32a [Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow startup time &c) (#26709) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-23 20:58:39 +08:00
wang.yuqi 3729ed00ba [Model] Add num_cached_tokens for PoolingRequestOutput (#27378) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-23 14:03:42 +08:00
Giancarlo DelfinandNick Hill 6644796bf4 [V1][spec decode] return logprobs for spec decoding (#26060) 
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-22 22:59:59 -07:00
dongbo910220andCyrus Leung a0003b56b0 [Chore] Separate out system utilities from vllm.utils (#27201) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-22 20:25:25 +00:00
Nicolò Lucchesi 4dfdb821c8 [P/D] Dynamic kv_output_aggregator collect size (#26734) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-22 18:07:58 +02:00
Nick Hill 647214f3d5 [V0 Deprecation] Remove V0 executors (#27142) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 11:09:37 -07:00
Cyrus Leung d31f7844f8 [Misc] Move utils to avoid conflicts with stdlib, and move tests (#27169) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-19 05:20:55 -07:00
iAmir97andiAmir97 7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
dongbo910220 8a297115e2 [Chore] Separate out hashing utilities from vllm.utils (#27151) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-19 11:09:38 +08:00
Tova Movshovitz 83e760c57d [V1][Metrics][Plugin] Add plugin support for custom StatLoggerBase implementations (#22456) 
Signed-off-by: tovam <tovam@pliops.com>
 2025-10-18 15:12:46 -07:00
Patrick von Platengemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Rui Qiao
 		b038d9c40c [Data-parallel] Allow DP>1 for world_size > num_gpus on node (8) (#26367) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-17 08:24:42 -07:00
Nick Hill fe3b9372ad [Core] Change execute_model_with_error_logging() to be a ctx manager (#27060) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-17 11:45:32 +08:00
Cyrus Leung 4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Cyrus Leung d2740fafbf [Chore] Separate out vllm.utils.collections (#26990) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 08:35:35 +00:00
Cyrus Leung f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
Cyrus Leung 828523ad8e [Chore] Separate out vllm.utils.async_utils (#26913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 15:33:00 +00:00
Cyrus Leung 136a17fe6e [Chore] Separate out vllm.utils.func (#26904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 13:03:58 +00:00
Jialin Ouyang 07ca70af8d [Core][Easy] Use envs.__getattr__ for all Unify to environment variable access (#26810) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-15 01:41:18 +00:00
Jialin Ouyang 380f17527c [Perf] Cache vllm.env.__getattr__ result to avoid recomputation (#26146) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-14 17:03:21 -04:00
Michael Goin 3e051bda82 [UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (#26732) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-13 18:12:52 -07:00
Lucia Fang 8317f72354 [Misc][DP] support customized aggregated logger for dp (#24354) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-10-13 17:45:59 -07:00
Wentao Ye e251e457c5 [Log] Optimize Startup Log (#26601) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-14 02:06:57 +08:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Chen Zhang 606b00e80f [bugfix][DCP] fix block_size of hash in DCP prefix caching (#26296) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-10 03:02:49 -07:00
Cyrus Leung ad430a67ca [Metrics] Log multi-modal cache stats and fix reset (#26285) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-10 01:45:55 -07:00
Rui Qiao 757fa4a4da [DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (#23849) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-09 19:53:43 -07:00
Cyrus Leung 4bdf7ac593 [Bugfix] Fix SHM cache initialization (#26427) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 02:48:04 -07:00
Nick Hill bb6d8c21f9 [Bugfix] Catch and log invalid token ids in detokenizer #2 (#26445) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-08 21:20:25 -07:00
Ayush Satyam 5e65d6b2ad fix[DP][v1]: Prevent hangs from mismatched worker configurations (#26218) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-10-07 22:55:08 -07:00
Cyrus Leung 391612e78b [Frontend] Consolidate tokenizer init code (#26276) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-06 09:34:52 +00:00
Harry Mellor 6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung d78fda7cda [Renderer] Move Processor out of LLMEngine (#26165) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 15:08:22 +00:00
Yang LiuandDarkLight1337 812b7f54a8 [Renderer] Move Processor out of AsyncLLM (#24138) 
Signed-off-by: Yang <lymailforjob@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 11:29:45 +00:00
HUIJONG JEONG 3e70e3d4d5 add(v1): RequestStatesStats to RequestOutput (#24947) 
Signed-off-by: huijjj <huijong.jeong@squeezebits.com>
 2025-10-03 08:56:25 +00:00
Cyrus Leung 0ad9951c41 [Input] Remove unused prompt field (#26097) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 00:23:21 -07:00
Nick Hill 169313b9f8 [Misc] Make handling of SamplingParams clearer in n>1 case (#26032) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-01 19:31:39 -07:00