 Taneem Ibrahim
|
d9eb4e344f
|
[Bugfix] Reject tokenless chat and audio streams (#54708)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-01 11:24:19 -04:00 |
|
|
|
9e905f7450
|
[Bugfix] Account for client queue time in serve benchmarks (#54136)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-01 03:40:51 -07:00 |
|
|
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
704f12aa1e
|
[Pooling] Report input throughput for batched requests (#53213)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-22 09:01:40 -04:00 |
|
|
Stan WozniakandThomas Parnell
|
63550515b4
|
[Bugfix] Correct prompt lengths for timed_traces benchmark (#45423)
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
|
2026-08-13 21:51:24 +00:00 |
|
|
Martin HickeyandHarry Mellor
|
275b43183c
|
[MyPy] Fix mypy for vllm/benchmarks (#39896)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-23 15:22:29 +00:00 |
|
|
Ekagra Ranjan
|
226ba9fc9e
|
[ASR] Add Long Audio benchmark and correctness test (#44587)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-06-12 04:11:16 +00:00 |
|
|
|
89c6a41001
|
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457)
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-09 23:59:18 -07:00 |
|
|
Thibault Castells
|
5d09f471f4
|
[Misc] Support interleaved custom image benchmark datasets (#43636)
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
|
2026-05-26 03:37:25 -07:00 |
|
|
Yufeng He
|
de111f3246
|
[Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732)
|
2026-04-16 12:01:25 -07:00 |
|
|
Matthias Gehre
|
1e4748c66a
|
[Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" (#38654)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-04-14 11:00:40 +00:00 |
|
|
Wentao Ye
|
be292b7c14
|
[Bug] Fix pooling model benchmark script (#36300)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-03-09 11:17:45 -04:00 |
|
|
Jee Jee Li
|
7291d1b288
|
[Bugfix] Fix kernel benchmark (#33752)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-02-22 21:18:08 -08:00 |
|
|
Tomas Ruiz
|
144d9b7cc8
|
[Benchmarks] Reduce ready checker log verbosity (#34349)
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-02-11 20:57:57 +00:00 |
|
|
mgazz
|
599e4335a4
|
Support benchmarking of Geospatial models (#33922)
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
|
2026-02-10 07:04:16 -08:00 |
|
|
Ekagra RanjanandNicolò Lucchesi
|
1d5922fade
|
[ASR] Fix audio benchmark and add RTFx metric (#32300)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
|
2026-02-09 10:02:37 +00:00 |
|
|
Huy Do
|
2647163674
|
Save startup benchmark results as a list of values (#33629)
Signed-off-by: Huy Do <huydhn@gmail.com>
|
2026-02-03 20:37:51 -08:00 |
|
|
Zack Yu
|
754a8ca942
|
fix: only include Authorization header when OPENAI_API_KEY is set (#33488)
Signed-off-by: zack041 <zackyu041@gmail.com>
|
2026-01-31 23:35:09 -08:00 |
|
|
Huy Do
|
9ca66ecc10
|
Indicate compile mode in the benchmark results (#32990)
Signed-off-by: Huy Do <huydhn@gmail.com>
|
2026-01-30 15:34:36 -05:00 |
|
|
Nick Hill
|
098b2d66fe
|
[Benchmark] Don't default to temperature==0 in vllm bench serve (#32723)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-22 10:03:15 +00:00 |
|
|
Tomas Ruiz
|
4a5299c93f
|
feat: spec decode with draft models (#24322)
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-01-19 16:05:46 -05:00 |
|
|
Cyrus Leung
|
b7030d962b
|
[Benchmark] Enable benchmark to run with encoding_format="bytes" (#27467)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-24 11:16:50 +00:00 |
|
|
Cyrus Leung
|
17838e50ef
|
[Benchmark] Use truncation by default for pooling benchmarks (#26992)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-16 16:02:39 +08:00 |
|
|
Maximilien de Bayser
|
fe3edb4cf0
|
Add support for the /rerank endpoint in vllm bench serve (#26602)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
|
2025-10-14 04:25:43 +00:00 |
|
|
Harry Mellor
|
8fcaaf6a16
|
Update Optional[x] -> x | None and Union[x, y] to x | y (#26633)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-12 09:51:31 -07:00 |
|
|
Cyrus Leung
|
5be7ca1b99
|
[Benchmark] Support Infinity API (#26641)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-12 01:45:32 +08:00 |
|
|
Cyrus Leung
|
44b9af5bb2
|
[Benchmark] Enable MM Embedding benchmarks (#26310)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-06 19:51:58 +00:00 |
|
|
Harry Mellor
|
d6953beb91
|
Convert formatting to use ruff instead of yapf + isort (#26247)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-05 07:06:22 -07:00 |
|
|
samzong
|
ce75e15373
|
refactor(benchmarks): add type annotations to wait_for_endpoint parameters (#25218)
Signed-off-by: samzong <samzong.lu@gmail.com>
|
2025-09-19 16:36:52 +00:00 |
|
|
Simon Mo
|
a904ea78ea
|
[benchmark] add peak throughput metrics and plot (#23867)
Signed-off-by: simon-mo <simon.mo@hey.com>
|
2025-09-17 22:30:02 -07:00 |
|
|
Clayton Coleman
|
bc636f21a6
|
[Benchmark] Allow arbitrary headers to be passed to benchmarked endpoints (#23937)
Signed-off-by: Clayton Coleman <smarterclayton@gmail.com>
|
2025-09-12 13:57:53 -07:00 |
|
|
Ming Yang
|
1823a00d67
|
[Misc] Support bench serve long context (#24373)
Signed-off-by: Ming Yang <minos.future@gmail.com>
|
2025-09-08 22:53:10 -07:00 |
|
|
Huy Do
|
6ace2f72b0
|
Fix writing benchmark results with tuple keys (#23633)
Signed-off-by: Huy Do <huydhn@gmail.com>
|
2025-08-26 19:16:09 +08:00 |
|
|
Jiangyun Zhu
|
3ecbb14b81
|
[Benchmarks] add benchmark for embedding models (#23000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2025-08-25 23:57:08 -07:00 |
|
|
Jared O'ConnellandCyrus Leung
|
31282401b6
|
[BugFix] Fix Python 3.9 Support (#23306)
Signed-off-by: Jared O'Connell <46976761+jaredoconnell@users.noreply.github.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2025-08-20 23:23:56 -07:00 |
|
|
hustxiayang
|
31436e8b4f
|
[Misc] Add request_id into benchmark_serve.py (#23065)
Signed-off-by: yangxia <yangxiast@gmail.com>
|
2025-08-19 08:32:18 +00:00 |
|
|
Breno Baldas Skuk
|
65a7917be4
|
Fix(benchmarks): allow multiple mm contents in OpenAI Chat Completion Benchmarks (#22534)
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
|
2025-08-10 09:03:15 -07:00 |
|
|
Seiji Eicher
|
6f5478298d
|
Use aiohttp connection pool for benchmarking (#21981)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
|
2025-08-03 19:23:32 -07:00 |
|
|
Ye (Charlotte) QiandRoger Wang
|
3f36c325fa
|
[Benchmark] Support ready check timeout in vllm bench serve (#21696)
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
|
2025-08-03 00:52:38 -07:00 |
|