9e905f7450
[Bugfix] Account for client queue time in serve benchmarks ( #54136 )
...
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com >
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:40:51 -07:00
Taneem Ibrahim and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
704f12aa1e
[Pooling] Report input throughput for batched requests ( #53213 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 09:01:40 -04:00
Stan Wozniak and Thomas Parnell
63550515b4
[Bugfix] Correct prompt lengths for timed_traces benchmark ( #45423 )
...
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-08-13 21:51:24 +00:00
Guan-Ming Chiu
63e78ce365
[Benchmark] Add probe requests to vllm bench serve ( #49611 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-08-01 22:10:56 +08:00
mgazz and Kevin H. Luu
a454a1dd25
[Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset ( #49180 )
...
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-07-24 04:30:19 -07:00
Martin Hickey and Harry Mellor
275b43183c
[MyPy] Fix mypy for vllm/benchmarks ( #39896 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-23 15:22:29 +00:00
eb28452b10
[Model] Add DiffusionGemma Support ( #45163 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Martin Kukla <martin.kukla@cantab.net >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Dipika Sikka <dsikka@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com >
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-06-11 22:17:35 -07:00
Harry Mellor
3cc9fecd58
Deprecated 1st generation Qwen and QwenVL models ( #45131 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-10 14:55:33 +00:00
89c6a41001
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads ( #42457 )
...
Signed-off-by: Li Zhang <lzhanga@amazon.com >
Co-authored-by: Li Zhang <lzhanga@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-06-09 23:59:18 -07:00
akii96
ac3409d162
[Benchmark] Auto-detect and correct client/server tokenizer mismatch for random dataset ( #44708 )
2026-06-08 06:10:20 -07:00
Albert Cheng
e0081ef8cf
[Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets ( #44244 )
...
Signed-off-by: Albert Cheng <albertching0112@gmail.com >
2026-06-02 22:49:51 -07:00
Animesh Trivedi
bfb9ebc211
[Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces ( #39795 )
...
Signed-off-by: Animesh Trivedi <Animesh.Trivedi@ibm.com >
2026-05-28 03:31:34 -07:00
noobHappylife
06a60d3dd0
Fix spec decode benchmark metrics ( #41916 )
...
Signed-off-by: noobhappylife <aratar1991@hotmail.com >
2026-05-07 09:23:21 -07:00
Sophie du Couédic and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
ff2c2bd80a
[Docs]Add documentation for bench serve visualization arguments ( #40539 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-23 15:48:29 -07:00
Talor Abramovich and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
7b1e0b07d0
[Bugfix] Fix dataset name and path argument validation bug in vllm bench serve ( #40288 )
...
Signed-off-by: talora <talora@nvidia.com >
Signed-off-by: Talor Abramovich <talor19@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-21 06:14:28 -07:00
Matthias Gehre
1e4748c66a
[Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" ( #38654 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-14 11:00:40 +00:00
RonaldBXu
c9d838fc33
Adding deterministic lora benchmarking to vLLM Bench ( #36057 )
...
Signed-off-by: Ubuntu <ubuntu@ip-172-31-43-201.ap-northeast-1.compute.internal >
Signed-off-by: Ronald Xu <ronaldxu@amazon.com >
2026-03-18 16:02:03 +00:00
Wentao Ye
be292b7c14
[Bug] Fix pooling model benchmark script ( #36300 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-09 11:17:45 -04:00
Cyrus Leung
7196348157
[Bugfix] Fix Qwen-VL tokenizer implementation ( #36140 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-05 08:07:19 -08:00
Sophie du Couédic and Cyrus Leung
02acd16861
[Benchmarks] Plot benchmark timeline and requests statistics ( #35220 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-02-26 02:17:43 -08:00
Cyrus Leung
3bcd494ef4
[Redo] Add --trust-remote-code to dataset bench args ( #34251 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-11 11:10:12 +08:00
mgazz
599e4335a4
Support benchmarking of Geospatial models ( #33922 )
...
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com >
2026-02-10 07:04:16 -08:00
Fan Yang and Fan Yang
a1946570d8
add --insecure arg to the vllm bench to skip TLS ( #34026 )
...
Signed-off-by: Fan Yang <yan9fan@meta.com >
Co-authored-by: Fan Yang <yan9fan@meta.com >
2026-02-10 22:23:52 +08:00
Cyrus Leung
998e2d91f8
Revert #34208 ( #34216 )
2026-02-09 23:59:04 -08:00
Cyrus Leung
f69b903b4c
[Bugfix] Add --trust-remote-code to dataset bench args ( #34208 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-09 22:37:50 -08:00
Ekagra Ranjan and Nicolò Lucchesi
1d5922fade
[ASR] Fix audio benchmark and add RTFx metric ( #32300 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2026-02-09 10:02:37 +00:00
Nick Hill
098b2d66fe
[Benchmark] Don't default to temperature==0 in vllm bench serve ( #32723 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-01-22 10:03:15 +00:00
Kebe
1bf1a34b19
[bench] add start_times field to vllm bench serve json result ( #32667 )
...
Signed-off-by: Kebe <mail@kebe7jun.com >
2026-01-22 07:10:14 +00:00
Nicolò Lucchesi
22970c1626
[Misc] Disable default --ready-check-timeout-sec extra call in vllm bench ( #30975 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-12 01:58:21 -08:00
d49899732e
[Spec Decode][UX] Add acceptance stats to vllm bench serve report ( #31739 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-01-06 21:21:42 +00:00
Elvir Crnčević
dba95378a6
Report error log after vllm bench serve ( #31808 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-01-06 20:24:19 +00:00
Ning Xie
3b8f31b362
[benchmark] use model card root instead of id ( #31329 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2025-12-26 10:55:56 +08:00
Ming Yang
3bb9561928
Revert "[bench] Support common prefix len config (for decode-only bench)" ( #31240 )
...
Signed-off-by: Ming Yang <minos.future@gmail.com >
2025-12-23 21:17:23 -08:00
Michael Goin
519ef9a911
[UX] Make vllm bench serve discover model by default and use --input-len ( #30816 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2025-12-17 01:55:30 -08:00
Vensen
add4b0ca44
[Bugfix][benchmarks] Fix input token calculation for rerank benchmark metrics ( #30596 )
...
Signed-off-by: vensen <vensenmu@gmail.com >
2025-12-14 14:57:15 +00:00
Matthew Bonanni
794a7875ee
[Misc] Consistent case for vllm bench serve results ( #30403 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2025-12-10 09:44:02 -08:00
e858bfe051
[Cleanup] Refactor profiling env vars into a CLI config ( #29912 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-09 13:29:33 -05:00
Rohan Potdar
40a046cd82
[Bugfix]: Fix TokenizerLike interface ( #30009 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2025-12-05 20:56:40 -08:00
Ming Yang
f16356fe36
[bench] Support common prefix len config (for decode-only bench) ( #29934 )
...
Signed-off-by: Ming Yang <minos.future@gmail.com >
2025-12-05 10:26:52 +00:00
Cyrus Leung
653591d5e7
[Chore] Move tokenizer initialization methods ( #29793 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-02 13:33:37 +08:00
Didier Durand
66d3d5422c
[Doc]: fixing typos in diverse files ( #29492 )
...
Signed-off-by: Didier Durand <durand.didier@gmail.com >
2025-11-27 07:15:50 -08:00
scottzh8 and Scott Zhang
3bc1175798
[Bugfix] Fix host and port join for ipv6 in bench serve ( #28679 )
...
Signed-off-by: Scott Zhang <scottzh@fb.com >
Co-authored-by: Scott Zhang <scottzh@fb.com >
2025-11-16 10:20:57 +00:00
Jialin Ouyang
b30372cbd0
[Perf] Move gc.freeze logic from EngineCoreProc to EngineCore for better coverage ( #27896 )
...
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2025-11-10 15:34:18 -08:00
Wentao Ye
4b1ff13221
[Feature] Default ignore_eos True for random dataset ( #28227 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2025-11-07 07:35:33 -05:00
Sophie du Couédic
a4398fbb5e
[Feature][Benchmarks] Support inf burstiness ( #26941 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
2025-11-03 18:33:17 +00:00
Seiji Eicher
b2e65cb4a7
[benchmark] Make request IDs unique across clients by default ( #27723 )
...
Signed-off-by: Seiji Eicher <seiji@anyscale.com >
2025-10-30 17:40:35 -07:00
Harry Mellor
6c9fdbf725
[Docs] Replace rst style double-backtick with md single-backtick ( #27091 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-10-17 02:47:34 -07:00
Tomas Ruiz
965c5f4914
vllm bench serve shows num of failed requests ( #26478 )
...
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2025-10-16 19:55:09 -07:00
kimbochen
013abde6ef
Adding Warmup to Benchmark Serving ( #26943 )
...
Signed-off-by: Kimbo Chen <chentenghung@gmail.com >
2025-10-16 12:44:32 -07:00
Cyrus Leung
334535b6fb
[Benchmark] Show E2EL by default for pooling models ( #27014 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-10-16 12:47:09 +00:00
