iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,919 Commits 3 Branches 0 Tags
1806d1adfc9b598bc6eb94de38a330aaad04c291
Commit Graph
209 Commits
Author SHA1 Message Date
yzong-rh 905b97adfa [Benchmark] Add num-warmup to vllm bench throughput (#43245) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-05-21 05:13:15 +00:00
pmaybankgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fadf5d332c add enqueue all option to throughput benchmark (#42975) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 20:16:02 -07:00
Sergei SkvortsovandMichael Goin 9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-15 13:01:21 -07:00
Yasmin Moslemandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 28ee78af54 Implement custom dataset class for ASR benchmarking (#41576) 
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 12:17:58 +08:00
noobHappylife 06a60d3dd0 Fix spec decode benchmark metrics (#41916) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-05-07 09:23:21 -07:00
Akim Tsvigun 894a02500b [Bench] Forward --seed to CustomDataset and CustomMMDataset shuffle (#40788) 
Signed-off-by: akimtsvigun <akimtsvigun@gmail.com>
 2026-05-04 00:39:10 +00:00
pmaybankandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 5b39b268f5 hf_name argument for vllm bench throughput CLI (#41012) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 12:57:58 +00:00
zhrrr ea74f701db Bugfix: fix SpecBench sample argument error (#40927) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2026-04-28 00:33:49 -07:00
Sophie du Couédicandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ff2c2bd80a [Docs]Add documentation for bench serve visualization arguments (#40539) 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-23 15:48:29 -07:00
Talor Abramovichandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 7b1e0b07d0 [Bugfix] Fix dataset name and path argument validation bug in vllm bench serve (#40288) 
Signed-off-by: talora <talora@nvidia.com>
Signed-off-by: Talor Abramovich <talor19@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-21 06:14:28 -07:00
Michael Goin 1948d0c467 [UX] Defer some imports on CLI paths to save ~2s (#40056) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-16 19:48:37 -07:00
Yufeng He de111f3246 [Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732) 2026-04-16 12:01:25 -07:00
Talor AbramovichandBenjamin Chislett 3cc328a4be [SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029) 
Signed-off-by: talora <talora@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-04-15 12:00:07 -04:00
Lucas Kabela c08f3b2a62 Measure encoder compile time seperate from llm backbone (#39240) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-14 10:52:49 -04:00
Matthias Gehre 1e4748c66a [Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" (#38654) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-04-14 11:00:40 +00:00
Julien Debache 893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-04-14 09:20:03 +00:00
nemanjaudovic 1a2c17634e [Bugfix] Add missing ASRDataset import and CLI args in benchmarks/throughput.py (#38114) 
Signed-off-by: nemanjaudovic <nudovic@amd.com>
 2026-04-08 13:53:53 +00:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Harry Mellor d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
Flora Feng 9040151fe1 [V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-20 11:31:43 +08:00
Harry Mellor 572b432913 Stop bench CLI from recursively casting all configs to dict (#37559) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-19 14:04:03 +00:00
RonaldBXu c9d838fc33 Adding deterministic lora benchmarking to vLLM Bench (#36057) 
Signed-off-by: Ubuntu <ubuntu@ip-172-31-43-201.ap-northeast-1.compute.internal>
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
 2026-03-18 16:02:03 +00:00
Ekagra Ranjan b5ca9c3557 [Models] Cohere ASR (#35809) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-03-17 21:04:17 +00:00
Isotr0py a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Wentao Ye be292b7c14 [Bug] Fix pooling model benchmark script (#36300) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-09 11:17:45 -04:00
Kunshang Ji 7eb524e64c refine vllm bench throughput --backend hf (#35971) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-07 02:10:33 +00:00
Cyrus Leung 7196348157 [Bugfix] Fix Qwen-VL tokenizer implementation (#36140) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-05 08:07:19 -08:00
Cyrus Leung 1dafb29f91 [Benchmark] Avoid unnecessary video download in MMVU (#35618) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-28 09:07:02 -08:00
flutist c68e69f144 custom dataset img support base64 (#35280) 
Signed-off-by: xjx <493337577@qq.com>
 2026-02-28 11:49:52 +00:00
Cyrus Leung 4292e3b807 [Benchmark] Improve UX of sweep scripts (#35600) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-28 00:36:02 -08:00
Cyrus Leung 24d6ea8afd [Benchmark] Rename SLA Finder to Workload Explorer (#35586) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-27 23:31:55 -08:00
Cyrus Leung fd68cd132b [Bugfix] Fixes for SLA finder (#35537) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-27 20:20:55 -08:00
Sophie du CouédicandCyrus Leung 02acd16861 [Benchmarks] Plot benchmark timeline and requests statistics (#35220) 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-26 02:17:43 -08:00
Cyrus Leung d3a51da92a [Benchmark] Simplify SLA scan (#35306) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-25 22:35:41 -08:00
Cyrus Leung 392645454b [Refactor] Decouple TimingContext from InputProcessingContext (#35083) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-23 14:15:50 +00:00
Jee Jee Li 7291d1b288 [Bugfix] Fix kernel benchmark (#33752) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-22 21:18:08 -08:00
Cyrus Leung cbd95a2dd1 [Benchmark] Use sns.relplot for plotting (#35027) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-21 20:26:48 -08:00
Cyrus Leung f74f1572ca [Benchmark] Improve benchmarks (#35012) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-21 10:31:58 +00:00
Cyrus Leung 73391a1baa [Renderer] Move InputPreprocessor into Renderer (1/2) (#34510) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-14 10:14:21 -08:00
Frank WangandRoger Wang b86bf4417e [Bugfix] Fix Random Dataset Prefix Length Inaccuracy (#33907) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-02-12 18:21:19 -08:00
Tomas Ruiz 144d9b7cc8 [Benchmarks] Reduce ready checker log verbosity (#34349) 
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2026-02-11 20:57:57 +00:00
Cyrus Leung 3bcd494ef4 [Redo] Add --trust-remote-code to dataset bench args (#34251) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-11 11:10:12 +08:00
mgazz 599e4335a4 Support benchmarking of Geospatial models (#33922) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
 2026-02-10 07:04:16 -08:00
Fan YangandFan Yang a1946570d8 add --insecure arg to the vllm bench to skip TLS (#34026) 
Signed-off-by: Fan Yang <yan9fan@meta.com>
Co-authored-by: Fan Yang <yan9fan@meta.com>
 2026-02-10 22:23:52 +08:00
Cyrus Leung 2c32558a3c [Bugfix] Fix --trust-remote-code conflict (#34218) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-10 00:29:10 -08:00
Cyrus Leung 998e2d91f8 Revert #34208 (#34216) 2026-02-09 23:59:04 -08:00
Andrew XiaandAndrew Xia 9608844f96 [responsesAPI] fix simpleContext streaming output_messages (#34188) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-02-09 22:53:07 -08:00
Cyrus Leung f69b903b4c [Bugfix] Add --trust-remote-code to dataset bench args (#34208) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-09 22:37:50 -08:00
Ekagra RanjanandNicolò Lucchesi 1d5922fade [ASR] Fix audio benchmark and add RTFx metric (#32300) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-09 10:02:37 +00:00