Michael Goin
|
1948d0c467
|
[UX] Defer some imports on CLI paths to save ~2s (#40056)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-04-16 19:48:37 -07:00 |
|
Yufeng He
|
de111f3246
|
[Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732)
|
2026-04-16 12:01:25 -07:00 |
|
 Talor AbramovichandBenjamin Chislett
|
3cc328a4be
|
[SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029)
Signed-off-by: talora <talora@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-04-15 12:00:07 -04:00 |
|
Lucas Kabela
|
c08f3b2a62
|
Measure encoder compile time seperate from llm backbone (#39240)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-04-14 10:52:49 -04:00 |
|
Matthias Gehre
|
1e4748c66a
|
[Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" (#38654)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-04-14 11:00:40 +00:00 |
|
Julien Debache
|
893b2affff
|
feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-04-14 09:20:03 +00:00 |
|
nemanjaudovic
|
1a2c17634e
|
[Bugfix] Add missing ASRDataset import and CLI args in benchmarks/throughput.py (#38114)
Signed-off-by: nemanjaudovic <nudovic@amd.com>
|
2026-04-08 13:53:53 +00:00 |
|
Cyrus Leung
|
ba2f0acc2d
|
[Misc] Reorganize inputs (#35182)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-25 10:22:54 -07:00 |
|
Harry Mellor
|
d215d1efca
|
[Mypy] Better fixes for the mypy issues in vllm/config (#37902)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-25 06:14:43 -07:00 |
|
Isotr0py
|
c7f98b4d0a
|
[Frontend] Remove librosa from audio dependency (#37058)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-03-21 11:36:15 +08:00 |
|
Flora Feng
|
9040151fe1
|
[V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-03-20 11:31:43 +08:00 |
|
Harry Mellor
|
572b432913
|
Stop bench CLI from recursively casting all configs to dict (#37559)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-19 14:04:03 +00:00 |
|
RonaldBXu
|
c9d838fc33
|
Adding deterministic lora benchmarking to vLLM Bench (#36057)
Signed-off-by: Ubuntu <ubuntu@ip-172-31-43-201.ap-northeast-1.compute.internal>
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
|
2026-03-18 16:02:03 +00:00 |
|
Ekagra Ranjan
|
b5ca9c3557
|
[Models] Cohere ASR (#35809)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-03-17 21:04:17 +00:00 |
|
Isotr0py
|
a836524d20
|
[Chore] Replace all base64 usages with faster pybase64 package (#37290)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-03-17 14:44:19 +00:00 |
|
Wentao Ye
|
be292b7c14
|
[Bug] Fix pooling model benchmark script (#36300)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-09 11:17:45 -04:00 |
|
Kunshang Ji
|
7eb524e64c
|
refine vllm bench throughput --backend hf (#35971)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-03-07 02:10:33 +00:00 |
|
Cyrus Leung
|
7196348157
|
[Bugfix] Fix Qwen-VL tokenizer implementation (#36140)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-05 08:07:19 -08:00 |
|
Cyrus Leung
|
1dafb29f91
|
[Benchmark] Avoid unnecessary video download in MMVU (#35618)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-28 09:07:02 -08:00 |
|
flutist
|
c68e69f144
|
custom dataset img support base64 (#35280)
Signed-off-by: xjx <493337577@qq.com>
|
2026-02-28 11:49:52 +00:00 |
|
Cyrus Leung
|
4292e3b807
|
[Benchmark] Improve UX of sweep scripts (#35600)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-28 00:36:02 -08:00 |
|
Cyrus Leung
|
24d6ea8afd
|
[Benchmark] Rename SLA Finder to Workload Explorer (#35586)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-27 23:31:55 -08:00 |
|
Cyrus Leung
|
fd68cd132b
|
[Bugfix] Fixes for SLA finder (#35537)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-27 20:20:55 -08:00 |
|
 Sophie du CouédicandCyrus Leung
|
02acd16861
|
[Benchmarks] Plot benchmark timeline and requests statistics (#35220)
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-02-26 02:17:43 -08:00 |
|
Cyrus Leung
|
d3a51da92a
|
[Benchmark] Simplify SLA scan (#35306)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-25 22:35:41 -08:00 |
|
Cyrus Leung
|
392645454b
|
[Refactor] Decouple TimingContext from InputProcessingContext (#35083)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-23 14:15:50 +00:00 |
|
Jee Jee Li
|
7291d1b288
|
[Bugfix] Fix kernel benchmark (#33752)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-02-22 21:18:08 -08:00 |
|
Cyrus Leung
|
cbd95a2dd1
|
[Benchmark] Use sns.relplot for plotting (#35027)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-21 20:26:48 -08:00 |
|
Cyrus Leung
|
f74f1572ca
|
[Benchmark] Improve benchmarks (#35012)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-21 10:31:58 +00:00 |
|
Cyrus Leung
|
73391a1baa
|
[Renderer] Move InputPreprocessor into Renderer (1/2) (#34510)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-02-14 10:14:21 -08:00 |
|
 Frank WangandRoger Wang
|
b86bf4417e
|
[Bugfix] Fix Random Dataset Prefix Length Inaccuracy (#33907)
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-02-12 18:21:19 -08:00 |
|
Tomas Ruiz
|
144d9b7cc8
|
[Benchmarks] Reduce ready checker log verbosity (#34349)
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-02-11 20:57:57 +00:00 |
|
Cyrus Leung
|
3bcd494ef4
|
[Redo] Add --trust-remote-code to dataset bench args (#34251)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-11 11:10:12 +08:00 |
|
mgazz
|
599e4335a4
|
Support benchmarking of Geospatial models (#33922)
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
|
2026-02-10 07:04:16 -08:00 |
|
 Fan YangandFan Yang
|
a1946570d8
|
add --insecure arg to the vllm bench to skip TLS (#34026)
Signed-off-by: Fan Yang <yan9fan@meta.com>
Co-authored-by: Fan Yang <yan9fan@meta.com>
|
2026-02-10 22:23:52 +08:00 |
|
Cyrus Leung
|
2c32558a3c
|
[Bugfix] Fix --trust-remote-code conflict (#34218)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-10 00:29:10 -08:00 |
|
Cyrus Leung
|
998e2d91f8
|
Revert #34208 (#34216)
|
2026-02-09 23:59:04 -08:00 |
|
 Andrew XiaandAndrew Xia
|
9608844f96
|
[responsesAPI] fix simpleContext streaming output_messages (#34188)
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
|
2026-02-09 22:53:07 -08:00 |
|
Cyrus Leung
|
f69b903b4c
|
[Bugfix] Add --trust-remote-code to dataset bench args (#34208)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-02-09 22:37:50 -08:00 |
|
 Ekagra RanjanandNicolò Lucchesi
|
1d5922fade
|
[ASR] Fix audio benchmark and add RTFx metric (#32300)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
|
2026-02-09 10:02:37 +00:00 |
|
Huy Do
|
2647163674
|
Save startup benchmark results as a list of values (#33629)
Signed-off-by: Huy Do <huydhn@gmail.com>
|
2026-02-03 20:37:51 -08:00 |
|
 Komal Kumar TeruandCyrus Leung
|
ba871fb788
|
[Misc] support arbitrary MM datasets in spec dec bench (#33486)
Signed-off-by: kkt-cohere <komal@cohere.com>
Signed-off-by: Komal Kumar Teru <162363718+kkt-cohere@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-02-02 08:49:48 +00:00 |
|
Luka Govedič
|
672023877b
|
Change defaults for vllm bench startup (#33489)
|
2026-01-31 23:46:01 -08:00 |
|
Zack Yu
|
754a8ca942
|
fix: only include Authorization header when OPENAI_API_KEY is set (#33488)
Signed-off-by: zack041 <zackyu041@gmail.com>
|
2026-01-31 23:35:09 -08:00 |
|
Cyrus Leung
|
79b6ec6aab
|
[Bugfix] Fix inconsistent handling of cache reset (#33481)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-01-31 20:23:41 -08:00 |
|
Huy Do
|
9ca66ecc10
|
Indicate compile mode in the benchmark results (#32990)
Signed-off-by: Huy Do <huydhn@gmail.com>
|
2026-01-30 15:34:36 -05:00 |
|
Harry Mellor
|
2eb673a088
|
Add flake8-implicit-str-concat rules to Ruff (#33191)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-01-28 04:56:10 +00:00 |
|
 Hiroken.andXingran Wang
|
6450b536a6
|
[Bugfix] Fix E2E latency calculation and add warmup support in mm_processor benchmark (#32646)
Signed-off-by: Hongjian Zhang <zhanghongjian@xiaohongshu.com>
Signed-off-by: Xingran Wang <wangxingran123456@outlook.com>
Signed-off-by: Hiroken. <105287758+HirokenOvo@users.noreply.github.com>
Co-authored-by: Xingran Wang <wangxingran123456@outlook.com>
|
2026-01-24 10:31:41 +00:00 |
|
 Reagan LeeandHiroken.
|
06b557ecd9
|
feat(benchmark): add encoder forward pass benchmarking to mm-processor (#31655)
Signed-off-by: Reagan <reaganjlee@gmail.com>
Signed-off-by: Reagan Lee <96998476+reaganjlee@users.noreply.github.com>
Co-authored-by: Hiroken. <105287758+HirokenOvo@users.noreply.github.com>
|
2026-01-24 08:24:44 +00:00 |
|
Isotr0py
|
243e78c20f
|
[Benchmark][Bugfix] Fix race condtion when starting server for sweep benchmark (#32927)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-01-23 12:11:18 +00:00 |
|