iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2339 Commits
Author SHA1 Message Date
Sage Moore 558db8083c [V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths (#13095) 2025-02-22 05:25:41 -08:00
youkaichao 2382ad29d1 [ci] fix linter (#13701) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 20:28:59 +08:00
youkaichao 3e472d882a [core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Cyrus Leung 7f6bae561c [CI/Build] Fix pre-commit errors (#13696) 2025-02-22 00:31:26 -08:00
Mark McLoughlin 2cb8c1540e [Metrics] Add --show-hidden-metrics-for-version CLI arg (#13295) 2025-02-22 00:20:45 -08:00
Mark McLoughlin 1cd981da4f [V1][Metrics] Support vllm:cache_config_info (#13299) 2025-02-22 00:20:00 -08:00
Jennifer ZhaoandRoger Wang da31b5333e [Bugfix] V1 Memory Profiling: V0 Sampler Integration without Rejection Sampler (#13594) 
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-02-22 00:08:29 -08:00
Lu Fang bb78fb318e [v1] Support allowed_token_ids in v1 Sampler (#13210) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-02-22 14:13:05 +08:00
Jun Duan 68d535ef44 [Misc] Capture and log the time of loading weights (#13666) 2025-02-21 22:06:34 -08:00
Lucas WilkinsonPatrick Hornsimon-moTyler Michael Smith
 		288cc6c234 [Attention] MLA with chunked prefill (#12639) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-02-21 15:30:12 -08:00
Nick Hill 31aa045c11 [V1][Sampler] Avoid an operation during temperature application (#13587) 2025-02-20 22:05:56 -08:00
Woosuk Kwon d3ea50113c [V1][Minor] Print KV cache size in token counts (#13596) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-20 09:24:31 -08:00
youkaichaoandcennn ba81163997 [core] add sleep and wake up endpoint and v1 support (#12987) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
 2025-02-20 12:41:17 +08:00
Nick Hill a4c402a756 [BugFix] Avoid error traceback in logs when V1 LLM terminates (#13565) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-20 00:49:01 +00:00
Nick Hill caf7ff4456 [V1][Core] Generic mechanism for handling engine utility (#13060) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-19 17:09:22 +08:00
Woosuk Kwon 4c82229898 [V1][Spec Decode] Optimize N-gram matching with Numba (#13365) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 13:19:58 -08:00
Nick Hill 30172b4947 [V1] Optimize handling of sampling metadata and req_ids list (#13244) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-18 12:15:33 -08:00
Woosuk Kwon 4fb8142a0e [V1][PP] Enable true PP with Ray executor (#13472) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-18 09:15:32 -08:00
youkaichao 932b51cedd [v1] fix parallel config rank (#13445) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-18 12:33:45 +08:00
Woosuk Kwon cd4a72a28d [V1][Spec decode] Move drafter to model runner (#13363) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-17 15:40:12 -08:00
Cody Yu 6ac485a953 [V1][PP] Fix intermediate tensor values (#13417) 
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-02-17 13:37:45 -08:00
Woosuk Kwon 4c21ce9eba [V1] Get input tokens from scheduler (#13339) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-17 11:01:07 -08:00
Jee Jee Li 2010f04c17 [V1][Misc] Avoid unnecessary log output (#13289) 2025-02-16 19:26:24 -08:00
Woosuk Kwon 69e1d23e1e [V1][BugFix] Clean up rejection sampler & Fix warning msg (#13362) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 12:25:29 -08:00
Woosuk Kwon e18227b04a [V1][PP] Cache Intermediate Tensors (#13353) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 10:02:27 -08:00
Woosuk Kwon 7b89386553 [V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-16 09:39:08 -08:00
Lily Liu 80f63a3966 [V1][Spec Decode] Ngram Spec Decode (#12193) 
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-15 18:05:11 -08:00
Cody Yu 9206b3d7ec [V1][PP] Run engine busy loop with batch queue (#13064) 2025-02-15 03:59:01 -08:00
Mark McLoughlin 2ad1bc7afe [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) 2025-02-15 03:56:19 -08:00
Woosuk Kwon 0c73026844 [V1][PP] Fix memory profiling in PP (#13315) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-14 20:17:25 -08:00
Nick Hill 6a854c7a2b [V1][Sampler] Don't apply temp for greedy-only (#13311) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-14 18:10:53 -08:00
AoyuandAoyu a12934d3ec [V1][Core] min_p sampling support (#13191) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
 2025-02-14 15:50:05 -08:00
Lu Fang 6224a9f620 Support logit_bias in v1 Sampler (#13079) 2025-02-14 04:34:59 -08:00
Nick Hill 085b7b2d6c [V1] Simplify GPUModelRunner._update_states check (#13265) 2025-02-14 04:33:43 -08:00
Alexander Matveev 45f90bcbba [WIP] TPU V1 Support Refactored (#13049) 2025-02-14 00:21:53 -08:00
Kero Liang b0ccfc565a [Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126) 2025-02-13 22:39:20 -08:00
Sage Moore ba59b78a9c [ROCm][V1] Add intial ROCm support to V1 (#12790) 2025-02-13 22:21:50 -08:00
Varun Sundar RabindranathandVarun Sundar Rabindranath cbc40128eb [V1] LoRA - Enable Serving Usecase (#12883) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-02-14 14:21:12 +08:00
Roger Wang dd5ede4440 [V1] Consolidate MM cache size to vllm.envs (#13239) 2025-02-13 20:19:03 -08:00
AoyuAoyuyoukaichao
 		2092a6fa7d [V1][Core] Add worker_base for v1 worker (#12816) 
Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-02-13 20:35:18 +08:00
Roger Wang fdcf64d3c6 [V1] Clarify input processing and multimodal feature caching logic (#13211) 2025-02-13 03:43:24 -08:00
Rui Qiao 9605c1256e [V1][core] Implement pipeline parallel on Ray (#12996) 2025-02-13 08:02:46 +00:00
Murali Andoorveedu 4c0d93f4b2 [V1][Bugfix] Copy encoder input ids to fix set iteration issue during VLM abort (#13173) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2025-02-12 12:58:11 -08:00
bnellnm f4d97e4fc2 [Bug] [V1] Try fetching stop_reason from EngineOutput before checking the request (#13108) 2025-02-12 02:39:16 -08:00
Mark McLoughlin 75e6e14516 [V1][Metrics] Add several request timing histograms (#12644) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-02-11 10:14:00 -05:00
Cody Yu 41c5dd45b9 [V1][Metrics] Add GPU prefix cache hit rate % gauge (#12592) 2025-02-11 08:27:25 +00:00
Woosuk Kwon 2ff4857678 [V1][Minor] Move scheduler outputs to a separate file (#13062) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-11 02:10:06 +00:00
Nick Hill 67c4637ccf [V1] Use msgpack for core request serialization (#12918) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-02-10 11:35:56 +08:00
Woosuk Kwon 24700c346b [V1] Cache uses_mrope in GPUModelRunner (#12969) 2025-02-08 15:32:32 -08:00
Woosuk Kwon 870c37481e [V1][Minor] Remove outdated comment (#12968) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-08 12:48:30 -08:00