iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1050 Commits
Author SHA1 Message Date
Woosuk Kwon 5801e49776 [V0 Deprecation] Remove MQLLMEngine (#25019) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-16 21:29:27 -07:00
Sage MooreLucas WilkinsonLucas Wilkinsonyewentao256Lucas WilkinsonRobert Shaw
 		567939953b [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Chen Brucelemon412Harry Mellor
 		7ea5c73ad7 [Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 10:55:16 +00:00
Woosuk Kwon 759ef49b15 Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> c4afdb69cc Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
Ce GaoandChauncey f4a948f33f [Frontend] Skip stop in reasoning content (#14550) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-09-15 06:04:55 +00:00
Didier Durand 41ae4a1eab [Doc]: fix typos in various files (#24798) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-13 00:43:33 -07:00
Nick HillRonald1995Robert Shaw
 		4fdd6f5cbf [Core] Support async scheduling with uniproc executor (#24219) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-12 16:34:28 -07:00
dongluw a5b84f1cbf [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-09-12 07:54:17 -07:00
RichardoMuMu HuaiYe ZhangBenjamin Bartelssimon-mo瑜琮Aaron Pham22quinn
 		40b6c9122b [V1] feat:add engine v1 tracing (#20372) 
Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Signed-off-by: Ye Zhang <zhysishu@gmail.com>
Signed-off-by: RichardoMu <44485717+RichardoMrMu@users.noreply.github.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Ye Zhang <zhysishu@gmail.com>
Co-authored-by: Benjamin Bartels <benjamin@bartels.dev>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: 瑜琮 <ly186375@antfin.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-11 17:10:39 -07:00
Boyuan FengandHarry Mellor 94e6b2d55f Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Harry Mellor 5f5271f1ee Move LoRAConfig from config/__init__.py to config/lora.py (#24644) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 11:01:38 +00:00
Harry Mellor d6249d0699 Fix typing for safetensors_load_strategy (#24641) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 10:41:39 +00:00
shengshiqi-googleandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 41329a0ff9 [Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469) 
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 23:10:01 -07:00
Guy Stone 8a894084d2 [Engine][Chore] use local variable and remove output var assignment (#24554) 
Signed-off-by: Guy Stone <guys@spotify.com>
 2025-09-10 23:05:42 -07:00
Hanjie QiuandMichael Goin dcb28a332b [Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) 
Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-10 15:31:10 -07:00
Xingyu Liuandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 9fb74c27a7 [Core] Support configuration parsing plugin (#24277) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 11:32:43 -07:00
pwschuurmanOmer DayanCyrus Leung
 		4377b1ae3b [Bugfix] Update Run:AI Model Streamer Loading Integration (#23845) 
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Signed-off-by: Peter Schuurman <psch@google.com>
Co-authored-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-09 21:37:17 -07:00
Zebing Lin 82dfb12e52 [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-09-08 21:34:37 -07:00
Didier Durand f4962a6d55 [Doc]: fix typos in Python comments (#24417) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-08 00:22:16 -07:00
Woosuk Kwon 4172235ab7 [V0 deprecation] Deprecate V0 Neuron backend (#21159) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-06 16:15:18 -07:00
yzdshongchaoyoukaichao
 		ac201a0eaf [Feature] Support Decode Context Parallel (DCP) for MLA (#23734) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-06 13:24:05 +08:00
Didier DurandRussell BryantWentao Ye
 		83609ca91d [Doc]: fix typos in Python comments (#24173) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-04 08:52:17 -07:00
Lucas WilkinsonMatthew BonanniMatthew Bonanni
 		402759d472 [Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
Didier Durand 02d411fdb2 [Doc]: fix typos in Python comments (#24115) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 21:14:07 -07:00
Mark McLoughlin 2417798471 [Metrics] Deprecate TPOT in favor of ITL (#24110) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-09-02 18:10:10 +00:00
Isotr0py d7fbc6ddac [Misc] Enable V1 FP16 inference on pre-Ampere GPUs (#24022) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-01 08:12:22 +00:00
Christian PintoandMax de Bayser 1cb39dbcdd [Misc] IO Processor plugins for pooling models (#22820) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-31 23:07:12 -07:00
Didier Durand 9701352e4b [Doc]: fix typos in Python comments (#24001) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-31 08:21:59 +00:00
Maximilien de BayserandWoosuk Kwon 2554b27baa [V0 Deprecation] Remove pooling model support in V0 (#23434) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-29 00:04:02 -07:00
Didier Durand d3da2eea54 [Doc]: fix typos in Python scripts (#23828) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-28 05:37:38 -07:00
Asaf Joseph Gardin 853c371fc3 [V1][Mamba] - Enable V1 by default for Mamba Models (#23650) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-08-27 20:53:30 +00:00
Harry Mellor 513c1fe255 Only run get_attr_docs if generating help text (#23723) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-27 13:55:12 +00:00
Cyrus Leung 69244e67e6 [Core] Use key-only cache for BaseMultiModalProcessor (#23018) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 14:19:13 +08:00
Hyogeun Oh (오효근)andHarry Mellor 730d0ac8b9 [Docs] Fix warnings in mkdocs build (#23649) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-26 18:19:23 +00:00
Cyrus Leung 50fede6634 [V1] Enable V1 for compute capability < 8.0 + FP32 (#23614) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-26 03:00:18 -07:00
汪志鹏 416f05929a [New Model]Donut model (#23229) 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-08-24 12:52:24 +00:00
rongfu.leng 1b9b16649c [Misc] update dict parse to EPLBConfig from json dumps to dict unpacking (#23305) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-08-24 08:06:34 +00:00
Didier Durand 22cf679aad [Doc]: fix various typos in multiple files (#23179) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-22 10:38:46 -07:00
Matthew Bonanni 19fe1a0510 [Kernel] Add FP8 support with FlashMLA backend (#22668) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-08-22 02:26:32 +00:00
22quinn 480bdf5a7b [Core] Support custom executor qualname (#23314) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-22 09:40:54 +08:00
Robert ShawandRobert Shaw c8e33c72c6 [V1] Remove unnecessary check for main thread (#23298) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-08-21 14:08:35 +00:00
22quinnNick HillWoosuk Kwon
 		f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
Michael Goin bbea1cefdd [CI Bugfix] Fix CI by fully removing --enable-prompt-adapter (#23284) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-20 17:18:12 -07:00
rongfu.lengandHarry Mellor 4fbda0b20c [Feature] use --eplb_config to set eplb param (#20562) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-20 14:07:28 -07:00
Cyrus Leung 5efd6905bc [CLI][Doc] Formalize --mm-encoder-tp-mode (#23190) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-20 23:42:28 +08:00
rongfu.leng 38217877aa [Fix] fix offline env use local mode path (#22526) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-08-20 13:34:49 +00:00
22quinn f7cf5b512e [Frontend] Add /collective_rpc API endpoint (#23075) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-19 17:29:32 +00:00
Nikhil Suryawanshi 78dba404ad [Hardware][IBM Z]Enable v1 for s390x and s390x dockerfile fixes (#22725) 
Signed-off-by: Nikhil Suryawanshi <suryawanshin74@gmail.com>
 2025-08-19 04:40:37 +00:00
afeldman-nmNick HillAndrew FeldmanHarry Mellor
 		bf7f470b22 [V1] Logits processors extensibility (#19912) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-16 12:59:17 -07:00