 Woosuk Kwon
|
9d5b4e4dea
|
[V1] Enable custom ops with piecewise CUDA graphs (#10228)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:58:07 -08:00 |
|
|
Woosuk Kwon
|
fe15729a2b
|
[V1] Use custom ops for piecewise CUDA graphs (#10227)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:26:48 -08:00 |
|
|
youkaichaoandWoosuk Kwon
|
330e82d34a
|
[v1][torch.compile] support managing cudagraph buffer (#10203)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:10:27 -08:00 |
|
|
Woosuk Kwon
|
d7a4f2207b
|
[V1] Do not use inductor for piecewise CUDA graphs (#10225)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:05:57 -08:00 |
|
|
Woosuk Kwon
|
f9dadfbee3
|
[V1] Fix detokenizer ports (#10224)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 10:42:07 -08:00 |
|
|
youkaichao
|
e6de9784d2
|
[core][distributed] add stateless process group (#10216)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-11 09:02:14 -08:00 |
|
|
Yangcheng Li
|
36fc439de0
|
[Doc] fix doc string typo in block_manager swap_out function (#10212)
|
2024-11-11 08:53:07 -08:00 |
|
|
|
874f551b36
|
[Metrics] add more metrics (#4464)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-12 00:17:38 +08:00 |
|
|
Isotr0py
|
2cebda42bb
|
[Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-11 12:37:58 +00:00 |
|
|
Roger Wang
|
5fb1f935b0
|
[V1] Allow tokenizer_mode and trust_remote_code for Detokenizer (#10211)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-11-11 18:01:18 +08:00 |
|
|
Jee Jee Li
|
36e4acd02a
|
[LoRA][Kernel] Remove the unused libentry module (#10214)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-11-11 09:43:23 +00:00 |
|
|
Isotr0py
|
58170d6503
|
[Hardware][CPU] Add embedding models support for CPU backend (#10193)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-11 08:54:28 +00:00 |
|
|
youkaichao
|
f89d18ff74
|
[6/N] pass whole config to inner model (#10205)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-11 06:41:46 +00:00 |
|
|
yansh97
|
ad9a78bf64
|
[Doc] Fix typo error in vllm/entrypoints/openai/cli_args.py (#10196)
|
2024-11-11 00:14:22 +00:00 |
|
|
youkaichao
|
73b9083e99
|
[misc] improve cloudpickle registration and tests (#10202)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-11 00:10:53 +00:00 |
|
|
Shawn Duandyoukaichao
|
20cf2f553c
|
[Misc] small fixes to function tracing file path (#9543)
Signed-off-by: Shawn Du <shawnd200@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
|
2024-11-10 15:21:06 -08:00 |
|
|
Cyrus Leung
|
51c2e1fcef
|
[CI/Build] Split up models tests (#10069)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-09 11:39:14 -08:00 |
|
|
Krishna MandalandDarkLight1337
|
b09895a618
|
[Frontend][Core] Override HF config.json via CLI (#5836)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-09 16:19:27 +00:00 |
|
|
cjackal
|
d88bff1b96
|
[Frontend] add add_request_id middleware (#9594)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2024-11-09 10:18:29 +00:00 |
|
|
Zhao Yingzhuo
|
9e37266420
|
bugfix: fix the bug that stream generate not work (#2756)
|
2024-11-09 10:09:48 +00:00 |
|
|
youkaichao
|
bd46357ad9
|
[bugfix] fix broken tests of mlp speculator (#10177)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-09 00:04:50 -08:00 |
|
|
youkaichao
|
1a95f10ee7
|
[5/N] pass the whole config to model (#9983)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-09 14:17:28 +08:00 |
|
|
Michael Goin
|
f83feccd7f
|
[Bugfix] Ignore GPTQ quantization of Qwen2-VL visual module (#10169)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-11-09 03:36:46 +00:00 |
|
|
Cyrus Leung
|
e0191a95d8
|
[0/N] Rename MultiModalInputs to MultiModalKwargs (#10040)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-09 11:31:02 +08:00 |
|
|
rasmith
|
127c07480e
|
[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
|
2024-11-08 19:59:22 -05:00 |
|
|
bnellnm
|
10b67d865d
|
[Bugfix] SymIntArrayRef expected to contain concrete integers (#10170)
Signed-off-by: Bill Nell <bill@neuralmagic.com>
|
2024-11-08 14:44:18 -08:00 |
|
|
Luka Govedičandyoukaichao
|
4f93dfe952
|
[torch.compile] Fuse RMSNorm with quant (#9138)
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
|
2024-11-08 21:20:08 +00:00 |
|
|
Florian Zimmermeister
|
e1b5a82179
|
Rename vllm.logging to vllm.logging_utils (#10134)
|
2024-11-08 20:53:24 +00:00 |
|
|
Woosuk Kwon
|
b5815c8413
|
[V1] Fix non-cudagraph op name (#10166)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-08 10:23:04 -08:00 |
|
|
sroy745
|
f6778620a9
|
Disable spec-decode + chunked-prefill for draft models with tensor parallelism > 1 (#10136)
Signed-off-by: Sourashis Roy <sroy@roblox.com>
|
2024-11-08 15:56:18 +00:00 |
|
|
Patrick von Platen
|
0535e5fe6c
|
Fix edge case Mistral tokenizer (#10152)
|
2024-11-08 15:42:27 +00:00 |
|
|
Cyrus Leung
|
b489fc3c91
|
[CI/Build] Update CPU tests to include all "standard" tests (#5481)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-08 23:30:04 +08:00 |
|
|
Roger Wang
|
208ce622c7
|
[V1]Enable APC by default only for text models (#10148)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-11-08 14:39:41 +00:00 |
|
|
Isotr0py
|
1ff4aed5bd
|
[Model] Expose size to Idefics3 as mm_processor_kwargs (#10146)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-08 09:56:58 +00:00 |
|
|
Yan Ma
|
f10797c0ce
|
[Bugfix][XPU] Fix xpu tp by introducing XpuCommunicator (#10144)
Signed-off-by: yan ma <yan.ma@intel.com>
|
2024-11-08 09:41:03 +00:00 |
|
|
Mengqing Cao
|
7371749d54
|
[Misc] Fix ImportError causing by triton (#9493)
|
2024-11-08 05:08:51 +00:00 |
|
|
Cody Yu
|
201fc07730
|
[V1] Prefix caching (take 2) (#9972)
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-11-07 17:34:44 -08:00 |
|
|
Woosuk Kwon
|
42b4f46b71
|
[V1] Add all_token_ids attribute to Request (#10135)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-07 17:08:24 -08:00 |
|
|
|
28b2877d30
|
Online video support for VLMs (#10020)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-07 20:25:59 +00:00 |
|
|
Nicolò Lucchesi
|
9d43afcc53
|
[Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2024-11-07 08:15:14 -08:00 |
|
|
Maximilien de Bayser
|
ae62fd17c0
|
[Frontend] Tool calling parser for Granite 3.0 models (#9027)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
|
2024-11-07 07:09:02 -08:00 |
|
|
Jiahao Li
|
999df95b4e
|
[Bugfix] Make image processor respect mm_processor_kwargs for Qwen2-VL (#10112)
Signed-off-by: Jiahao Li <liplus17@163.com>
|
2024-11-07 10:50:44 +00:00 |
|
|
Lei Yang
|
0dfba97b42
|
[Frontend] Fix multiple values for keyword argument error (#10075) (#10076)
Signed-off-by: Lei <ylxx@live.com>
|
2024-11-07 09:07:19 +00:00 |
|
|
Flávia BéoandMax de Bayser
|
aa9078fa03
|
Adds method to read the pooling types from model's files (#9506)
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
|
2024-11-07 08:42:40 +00:00 |
|
|
Hanzhi Zhou
|
6192e9b8fe
|
[Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030)
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
|
2024-11-06 23:50:47 -08:00 |
|
|
Cyrus Leung
|
db7db4aab9
|
[Misc] Consolidate ModelConfig code related to HF config (#10104)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-07 06:00:21 +00:00 |
|
|
Nick Hill
|
1fa020c539
|
[V1][BugFix] Fix Generator construction in greedy + seed case (#10097)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2024-11-07 05:06:57 +00:00 |
|
|
Li, Jiang
|
a4b3e0c1e9
|
[Hardware][CPU] Update torch 2.5 (#9911)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2024-11-07 04:43:08 +00:00 |
|
|
Nick Hill
|
29862b884b
|
[Frontend] Adjust try/except blocks in API impl (#10056)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2024-11-06 20:07:51 -08:00 |
|
|
Yan MaandKunshang Ji
|
d3859f1891
|
[Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (#9823)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2024-11-06 17:29:03 -08:00 |
|