Tyrone
bafb263664
[Bugfix][Benchmarks] Name the real cause when the /tokenize alignment probe fails ( #59888 )
2026-10-17 14:47:52 -04:00
Harshil and Cyrus Leung
c54dca96bd
[Bugfix][Benchmark] Fix probe latency and duration in bench serve ( #60511 )
...
Signed-off-by: QHarshil <harshil_c@hotmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 05:50:30 +00:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Wentao Ye
3e182185aa
[Deprecation] Deprecate sonet dataset as scheduled ( #60129 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-06 08:15:12 -04:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
af7f9488c2
[Benchmark] Add Responses API backend to vllm bench serve ( #54628 )
...
Signed-off-by: QHarshil <harshil_c@hotmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-28 05:04:11 +00:00
Harry Mellor and Claude Opus 5.5
d636456b1f
[Frontend] Remove the slow tokenizer mode ( #58545 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 08:48:28 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Taneem Ibrahim and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8e08cef46e
[Bugfix] Redact credentials from benchmark logs ( #56662 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-14 18:58:33 -04:00
Taneem Ibrahim
3284af6bf1
[Bugfix] Reject 0 or non-positive max concurrency ( #54887 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-04 15:59:29 -04:00
9e905f7450
[Bugfix] Account for client queue time in serve benchmarks ( #54136 )
...
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com >
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:40:51 -07:00
Taneem Ibrahim and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
704f12aa1e
[Pooling] Report input throughput for batched requests ( #53213 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 09:01:40 -04:00
Stan Wozniak and Thomas Parnell
63550515b4
[Bugfix] Correct prompt lengths for timed_traces benchmark ( #45423 )
...
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-08-13 21:51:24 +00:00
Guan-Ming Chiu
63e78ce365
[Benchmark] Add probe requests to vllm bench serve ( #49611 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-08-01 22:10:56 +08:00
mgazz and Kevin H. Luu
a454a1dd25
[Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset ( #49180 )
...
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-07-24 04:30:19 -07:00
Martin Hickey and Harry Mellor
275b43183c
[MyPy] Fix mypy for vllm/benchmarks ( #39896 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-23 15:22:29 +00:00
eb28452b10
[Model] Add DiffusionGemma Support ( #45163 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Martin Kukla <martin.kukla@cantab.net >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Dipika Sikka <dsikka@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com >
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-06-11 22:17:35 -07:00
Harry Mellor
3cc9fecd58
Deprecated 1st generation Qwen and QwenVL models ( #45131 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-10 14:55:33 +00:00
89c6a41001
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads ( #42457 )
...
Signed-off-by: Li Zhang <lzhanga@amazon.com >
Co-authored-by: Li Zhang <lzhanga@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-06-09 23:59:18 -07:00
akii96
ac3409d162
[Benchmark] Auto-detect and correct client/server tokenizer mismatch for random dataset ( #44708 )
2026-06-08 06:10:20 -07:00
Albert Cheng
e0081ef8cf
[Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets ( #44244 )
...
Signed-off-by: Albert Cheng <albertching0112@gmail.com >
2026-06-02 22:49:51 -07:00
Animesh Trivedi
bfb9ebc211
[Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces ( #39795 )
...
Signed-off-by: Animesh Trivedi <Animesh.Trivedi@ibm.com >
2026-05-28 03:31:34 -07:00
noobHappylife
06a60d3dd0
Fix spec decode benchmark metrics ( #41916 )
...
Signed-off-by: noobhappylife <aratar1991@hotmail.com >
2026-05-07 09:23:21 -07:00
Sophie du Couédic and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
ff2c2bd80a
[Docs]Add documentation for bench serve visualization arguments ( #40539 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-23 15:48:29 -07:00
Talor Abramovich and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
7b1e0b07d0
[Bugfix] Fix dataset name and path argument validation bug in vllm bench serve ( #40288 )
...
Signed-off-by: talora <talora@nvidia.com >
Signed-off-by: Talor Abramovich <talor19@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-21 06:14:28 -07:00
Matthias Gehre
1e4748c66a
[Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" ( #38654 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-14 11:00:40 +00:00
RonaldBXu
c9d838fc33
Adding deterministic lora benchmarking to vLLM Bench ( #36057 )
...
Signed-off-by: Ubuntu <ubuntu@ip-172-31-43-201.ap-northeast-1.compute.internal >
Signed-off-by: Ronald Xu <ronaldxu@amazon.com >
2026-03-18 16:02:03 +00:00
Wentao Ye
be292b7c14
[Bug] Fix pooling model benchmark script ( #36300 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-09 11:17:45 -04:00
Cyrus Leung
7196348157
[Bugfix] Fix Qwen-VL tokenizer implementation ( #36140 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-05 08:07:19 -08:00
Sophie du Couédic and Cyrus Leung
02acd16861
[Benchmarks] Plot benchmark timeline and requests statistics ( #35220 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-02-26 02:17:43 -08:00
Cyrus Leung
3bcd494ef4
[Redo] Add --trust-remote-code to dataset bench args ( #34251 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-11 11:10:12 +08:00
mgazz
599e4335a4
Support benchmarking of Geospatial models ( #33922 )
...
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com >
2026-02-10 07:04:16 -08:00
Fan Yang and Fan Yang
a1946570d8
add --insecure arg to the vllm bench to skip TLS ( #34026 )
...
Signed-off-by: Fan Yang <yan9fan@meta.com >
Co-authored-by: Fan Yang <yan9fan@meta.com >
2026-02-10 22:23:52 +08:00
Cyrus Leung
998e2d91f8
Revert #34208 ( #34216 )
2026-02-09 23:59:04 -08:00
Cyrus Leung
f69b903b4c
[Bugfix] Add --trust-remote-code to dataset bench args ( #34208 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-09 22:37:50 -08:00
Ekagra Ranjan and Nicolò Lucchesi
1d5922fade
[ASR] Fix audio benchmark and add RTFx metric ( #32300 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2026-02-09 10:02:37 +00:00
Nick Hill
098b2d66fe
[Benchmark] Don't default to temperature==0 in vllm bench serve ( #32723 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-01-22 10:03:15 +00:00
Kebe
1bf1a34b19
[bench] add start_times field to vllm bench serve json result ( #32667 )
...
Signed-off-by: Kebe <mail@kebe7jun.com >
2026-01-22 07:10:14 +00:00
Nicolò Lucchesi
22970c1626
[Misc] Disable default --ready-check-timeout-sec extra call in vllm bench ( #30975 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-01-12 01:58:21 -08:00
d49899732e
[Spec Decode][UX] Add acceptance stats to vllm bench serve report ( #31739 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-01-06 21:21:42 +00:00
Elvir Crnčević
dba95378a6
Report error log after vllm bench serve ( #31808 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
2026-01-06 20:24:19 +00:00
Ning Xie
3b8f31b362
[benchmark] use model card root instead of id ( #31329 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2025-12-26 10:55:56 +08:00
Ming Yang
3bb9561928
Revert "[bench] Support common prefix len config (for decode-only bench)" ( #31240 )
...
Signed-off-by: Ming Yang <minos.future@gmail.com >
2025-12-23 21:17:23 -08:00
Michael Goin
519ef9a911
[UX] Make vllm bench serve discover model by default and use --input-len ( #30816 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2025-12-17 01:55:30 -08:00
Vensen
add4b0ca44
[Bugfix][benchmarks] Fix input token calculation for rerank benchmark metrics ( #30596 )
...
Signed-off-by: vensen <vensenmu@gmail.com >
2025-12-14 14:57:15 +00:00
Matthew Bonanni
794a7875ee
[Misc] Consistent case for vllm bench serve results ( #30403 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2025-12-10 09:44:02 -08:00
e858bfe051
[Cleanup] Refactor profiling env vars into a CLI config ( #29912 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-09 13:29:33 -05:00
Rohan Potdar
40a046cd82
[Bugfix]: Fix TokenizerLike interface ( #30009 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2025-12-05 20:56:40 -08:00
Ming Yang
f16356fe36
[bench] Support common prefix len config (for decode-only bench) ( #29934 )
...
Signed-off-by: Ming Yang <minos.future@gmail.com >
2025-12-05 10:26:52 +00:00