 Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
704f12aa1e
|
[Pooling] Report input throughput for batched requests (#53213)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-22 09:01:40 -04:00 |
|
|
Stan WozniakandThomas Parnell
|
63550515b4
|
[Bugfix] Correct prompt lengths for timed_traces benchmark (#45423)
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
|
2026-08-13 21:51:24 +00:00 |
|
|
Jared Wen
|
e07532b035
|
[MISC][Bench] refactor throughput and reuse serve's get samples (#50981)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
|
2026-08-06 10:31:23 +00:00 |
|
|
mgazzandKevin H. Luu
|
a454a1dd25
|
[Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset (#49180)
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-07-24 04:30:19 -07:00 |
|
|
almayne
|
d1f1d86797
|
[Bugfix] Re-enable benchmarking of librispeech dataset. (#47033)
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
|
2026-07-08 16:19:26 +00:00 |
|
|
Michał Ganczarenko
|
c8bdcc0116
|
[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
|
2026-07-01 15:42:27 +00:00 |
|
|
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
|
Martin HickeyandHarry Mellor
|
275b43183c
|
[MyPy] Fix mypy for vllm/benchmarks (#39896)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-23 15:22:29 +00:00 |
|
|
Ekagra Ranjan
|
226ba9fc9e
|
[ASR] Add Long Audio benchmark and correctness test (#44587)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-06-12 04:11:16 +00:00 |
|
|
Harry Mellor
|
03878d1c22
|
Deprecations for v0.23 and v0.24 (#44992)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-11 14:35:38 +00:00 |
|
|
|
89c6a41001
|
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457)
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-09 23:59:18 -07:00 |
|
|
Albert Cheng
|
e0081ef8cf
|
[Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets (#44244)
Signed-off-by: Albert Cheng <albertching0112@gmail.com>
|
2026-06-02 22:49:51 -07:00 |
|
|
XiaoZ
|
53fa09d085
|
[Misc] Support local image encoding in benchmarks (#43843)
Signed-off-by: xiaoz <Sukra1@outlook.com>
|
2026-06-02 15:15:06 +00:00 |
|
|
LucainandClaude Opus 4.6
|
11dfa3169d
|
Add vLLM library info to Hugging Face Hub requests (#43857)
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-05-29 14:04:58 +00:00 |
|
|
Julien Denize
|
02606b0b09
|
[BUGFIX] Multimodal benchmark with MistralTokenizer (#42965)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
|
2026-05-28 05:36:24 -07:00 |
|
|
Animesh Trivedi
|
bfb9ebc211
|
[Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces (#39795)
Signed-off-by: Animesh Trivedi <Animesh.Trivedi@ibm.com>
|
2026-05-28 03:31:34 -07:00 |
|
|
Thibault Castells
|
5d09f471f4
|
[Misc] Support interleaved custom image benchmark datasets (#43636)
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
|
2026-05-26 03:37:25 -07:00 |
|
|
Sergei SkvortsovandMichael Goin
|
9a7a273dfe
|
Add HumanEval and GSM8K benchmarks to datasets (#42648)
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-05-15 13:01:21 -07:00 |
|
|
Yasmin Moslemandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
28ee78af54
|
Implement custom dataset class for ASR benchmarking (#41576)
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-12 12:17:58 +08:00 |
|
|
Akim Tsvigun
|
894a02500b
|
[Bench] Forward --seed to CustomDataset and CustomMMDataset shuffle (#40788)
Signed-off-by: akimtsvigun <akimtsvigun@gmail.com>
|
2026-05-04 00:39:10 +00:00 |
|
|
zhrrr
|
ea74f701db
|
Bugfix: fix SpecBench sample argument error (#40927)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-04-28 00:33:49 -07:00 |
|
|
Talor Abramovichandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
7b1e0b07d0
|
[Bugfix] Fix dataset name and path argument validation bug in vllm bench serve (#40288)
Signed-off-by: talora <talora@nvidia.com>
Signed-off-by: Talor Abramovich <talor19@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-21 06:14:28 -07:00 |
|
|
Talor AbramovichandBenjamin Chislett
|
3cc328a4be
|
[SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029)
Signed-off-by: talora <talora@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-04-15 12:00:07 -04:00 |
|
|
Julien Debache
|
893b2affff
|
feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-04-14 09:20:03 +00:00 |
|