Wentao Ye
|
276fbcff27
|
[Deprecation] Remove scheduled deprecated items (#60923)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-10 15:45:33 -04:00 |
|
 sungbin1015andFlora Feng
|
2ddde19e8e
|
[Frontend] Support min_p in the Responses API (#48084)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 20:38:17 +00:00 |
|
 Harry MellorandClaude Opus 5.5
|
21d9758459
|
Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 02:30:52 -07:00 |
|
Wentao Ye
|
3e182185aa
|
[Deprecation] Deprecate sonet dataset as scheduled (#60129)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-06 08:15:12 -04:00 |
|
 czhu-cohereandaoshen02
|
6fddc9b21e
|
[Profiler] Add per-session profiling controls to Python and Rust frontends (#57875)
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-06 05:21:19 +00:00 |
|
 Raphaël RiallandandSimon Veitner
|
c32495d58f
|
[watermark] compatibility validation (#56801)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
|
2026-10-05 14:37:13 -04:00 |
|
 Prudhvi VudaandOpenAI Codex
|
ea106db806
|
[Bugfix][Bench] Clean up synthetic video files and writers (#59529)
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-02 02:51:34 +00:00 |
|
Louie Tsai
|
1b77cc3d67
|
[Benchmark] Add sweep warmup and failure recovery (#57305)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
|
2026-09-30 11:16:41 +08:00 |
|
 
|
af7f9488c2
|
[Benchmark] Add Responses API backend to vllm bench serve (#54628)
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-09-28 05:04:11 +00:00 |
|
Yashasvi Chaurasia
|
f83ea61cfe
|
[Benchmark] Record model_id in bench latency/throughput --output-json (#58112)
Signed-off-by: yashasvi <yashasvi@ibm.com>
|
2026-09-25 16:43:37 +00:00 |
|
 Harry MellorandClaude Opus 5.5
|
d636456b1f
|
[Frontend] Remove the slow tokenizer mode (#58545)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 08:48:28 -07:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Taneem Ibrahim
|
64856080b0
|
[Bugfix] Measure complete pooling responses (#56760)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-15 09:21:05 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
8e08cef46e
|
[Bugfix] Redact credentials from benchmark logs (#56662)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 18:58:33 -04:00 |
|
 SurajandClaude Opus 5
|
93e051e290
|
[Bugfix][Benchmark] Make streaming TTFT/E2E latency accounting consistent across endpoints (#55508)
Signed-off-by: surajm20061998 <surajm20061998@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-13 05:20:54 +00:00 |
|
Nicolò Lucchesi
|
9dcab80215
|
[Bugfix][Bench] Fix bench mm-processor crash in shared request sampling (#56300)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-11 16:53:04 +08:00 |
|
Taneem Ibrahim
|
3284af6bf1
|
[Bugfix] Reject 0 or non-positive max concurrency (#54887)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-04 15:59:29 -04:00 |
|
Taneem Ibrahim
|
d9eb4e344f
|
[Bugfix] Reject tokenless chat and audio streams (#54708)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-09-01 11:24:19 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
9e905f7450
|
[Bugfix] Account for client queue time in serve benchmarks (#54136)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-01 03:40:51 -07:00 |
|
Michał Ganczarenko
|
d1922cb5a7
|
[Bugfix] Restore multimodal support on the plain "vllm" throughput backend (#52168)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
|
2026-08-28 10:28:25 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
704f12aa1e
|
[Pooling] Report input throughput for batched requests (#53213)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-22 09:01:40 -04:00 |
|
wang.yuqi
|
eac636a7fa
|
[Frontend] Move api_server.py out openai folder (#52131)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-08-19 09:43:46 +00:00 |
|
 Stan WozniakandThomas Parnell
|
63550515b4
|
[Bugfix] Correct prompt lengths for timed_traces benchmark (#45423)
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
|
2026-08-13 21:51:24 +00:00 |
|
鐘天楽
|
a367cdbca0
|
[Profiler] Add minimal Triton Proton profiling backend (#48789)
Signed-off-by: Luossu <zhluosuu@outlook.com>
|
2026-08-11 14:51:57 -07:00 |
|
Harry Mellor
|
83ad767eed
|
[CI] fix docs on main (#51539)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-08-09 09:46:12 +00:00 |
|
Jared Wen
|
e07532b035
|
[MISC][Bench] refactor throughput and reuse serve's get samples (#50981)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
|
2026-08-06 10:31:23 +00:00 |
|
Guan-Ming Chiu
|
63e78ce365
|
[Benchmark] Add probe requests to vllm bench serve (#49611)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-08-01 22:10:56 +08:00 |
|
 mgazzandKevin H. Luu
|
a454a1dd25
|
[Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset (#49180)
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-07-24 04:30:19 -07:00 |
|
almayne
|
d1f1d86797
|
[Bugfix] Re-enable benchmarking of librispeech dataset. (#47033)
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
|
2026-07-08 16:19:26 +00:00 |
|
Michał Ganczarenko
|
c8bdcc0116
|
[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
|
2026-07-01 15:42:27 +00:00 |
|
 
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)  
|
489abadfb8
|
feat: support to OpenMOSS-Team (#44124)
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa19 <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-24 00:08:13 -07:00 |
|
 Martin HickeyandHarry Mellor
|
275b43183c
|
[MyPy] Fix mypy for vllm/benchmarks (#39896)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-23 15:22:29 +00:00 |
|
        
|
eb28452b10
|
[Model] Add DiffusionGemma Support (#45163)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
|
2026-06-11 22:17:35 -07:00 |
|
Ekagra Ranjan
|
226ba9fc9e
|
[ASR] Add Long Audio benchmark and correctness test (#44587)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-06-12 04:11:16 +00:00 |
|
Harry Mellor
|
03878d1c22
|
Deprecations for v0.23 and v0.24 (#44992)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-11 14:35:38 +00:00 |
|
Harry Mellor
|
3cc9fecd58
|
Deprecated 1st generation Qwen and QwenVL models (#45131)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-10 14:55:33 +00:00 |
|
  
|
89c6a41001
|
[Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457)
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-09 23:59:18 -07:00 |
|
akii96
|
ac3409d162
|
[Benchmark] Auto-detect and correct client/server tokenizer mismatch for random dataset (#44708)
|
2026-06-08 06:10:20 -07:00 |
|
wang.yuqi
|
d01d0b4646
|
[Frontend] Consolidate online serving utils. (#44479)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-04 06:49:31 +00:00 |
|
Albert Cheng
|
e0081ef8cf
|
[Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets (#44244)
Signed-off-by: Albert Cheng <albertching0112@gmail.com>
|
2026-06-02 22:49:51 -07:00 |
|
XiaoZ
|
53fa09d085
|
[Misc] Support local image encoding in benchmarks (#43843)
Signed-off-by: xiaoz <Sukra1@outlook.com>
|
2026-06-02 15:15:06 +00:00 |
|
 LucainandClaude Opus 4.6
|
11dfa3169d
|
Add vLLM library info to Hugging Face Hub requests (#43857)
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-05-29 14:04:58 +00:00 |
|
Julien Denize
|
02606b0b09
|
[BUGFIX] Multimodal benchmark with MistralTokenizer (#42965)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
|
2026-05-28 05:36:24 -07:00 |
|
Animesh Trivedi
|
bfb9ebc211
|
[Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces (#39795)
Signed-off-by: Animesh Trivedi <Animesh.Trivedi@ibm.com>
|
2026-05-28 03:31:34 -07:00 |
|
Thibault Castells
|
5d09f471f4
|
[Misc] Support interleaved custom image benchmark datasets (#43636)
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
|
2026-05-26 03:37:25 -07:00 |
|
yzong-rh
|
905b97adfa
|
[Benchmark] Add num-warmup to vllm bench throughput (#43245)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-05-21 05:13:15 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) ![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)
|
fadf5d332c
|
add enqueue all option to throughput benchmark (#42975)
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-19 20:16:02 -07:00 |
|
 Sergei SkvortsovandMichael Goin
|
9a7a273dfe
|
Add HumanEval and GSM8K benchmarks to datasets (#42648)
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-05-15 13:01:21 -07:00 |
|