iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
238 Commits
Author SHA1 Message Date
Taneem Ibrahim d9eb4e344f [Bugfix] Reject tokenless chat and audio streams (#54708) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-01 11:24:19 -04:00
maithilijoshi20maithilijoshi20mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		9e905f7450 [Bugfix] Account for client queue time in serve benchmarks (#54136) 
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 03:40:51 -07:00
Michał Ganczarenko d1922cb5a7 [Bugfix] Restore multimodal support on the plain "vllm" throughput backend (#52168) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-08-28 10:28:25 +00:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 704f12aa1e [Pooling] Report input throughput for batched requests (#53213) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 09:01:40 -04:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
Stan WozniakandThomas Parnell 63550515b4 [Bugfix] Correct prompt lengths for timed_traces benchmark (#45423) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-08-13 21:51:24 +00:00
鐘天楽 a367cdbca0 [Profiler] Add minimal Triton Proton profiling backend (#48789) 
Signed-off-by: Luossu <zhluosuu@outlook.com>
 2026-08-11 14:51:57 -07:00
Harry Mellor 83ad767eed [CI] fix docs on main (#51539) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-09 09:46:12 +00:00
Jared Wen e07532b035 [MISC][Bench] refactor throughput and reuse serve's get samples (#50981) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-08-06 10:31:23 +00:00
Guan-Ming Chiu 63e78ce365 [Benchmark] Add probe requests to vllm bench serve (#49611) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-08-01 22:10:56 +08:00
mgazzandKevin H. Luu a454a1dd25 [Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset (#49180) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-07-24 04:30:19 -07:00
almayne d1f1d86797 [Bugfix] Re-enable benchmarking of librispeech dataset. (#47033) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
 2026-07-08 16:19:26 +00:00
Michał Ganczarenko c8bdcc0116 [Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-07-01 15:42:27 +00:00
Benjamin ChislettGiancarlo Delfinmgoin
 		f5a8d73377 [Spec Decode] DSpark (#46995) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-07-01 08:30:24 -07:00
hurukawaOpenAI CodexRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		489abadfb8 feat: support to OpenMOSS-Team (#44124) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa19 <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 00:08:13 -07:00
Martin HickeyandHarry Mellor 275b43183c [MyPy] Fix mypy for vllm/benchmarks (#39896) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-23 15:22:29 +00:00
Lucas WilkinsonMartin KuklaMatthew BonanniDipika SikkaNickLucchejiahancAlec KohlhoffPorras HuangClaudescoootscooob
 		eb28452b10 [Model] Add DiffusionGemma Support (#45163) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
 2026-06-11 22:17:35 -07:00
Ekagra Ranjan 226ba9fc9e [ASR] Add Long Audio benchmark and correctness test (#44587) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-06-12 04:11:16 +00:00
Harry Mellor 03878d1c22 Deprecations for v0.23 and v0.24 (#44992) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-11 14:35:38 +00:00
Harry Mellor 3cc9fecd58 Deprecated 1st generation Qwen and QwenVL models (#45131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-10 14:55:33 +00:00
LiLi ZhangClaudeChauncey
 		89c6a41001 [Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457) 
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-06-09 23:59:18 -07:00
akii96 ac3409d162 [Benchmark] Auto-detect and correct client/server tokenizer mismatch for random dataset (#44708) 2026-06-08 06:10:20 -07:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-04 06:49:31 +00:00
Albert Cheng e0081ef8cf [Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets (#44244) 
Signed-off-by: Albert Cheng <albertching0112@gmail.com>
 2026-06-02 22:49:51 -07:00
XiaoZ 53fa09d085 [Misc] Support local image encoding in benchmarks (#43843) 
Signed-off-by: xiaoz <Sukra1@outlook.com>
 2026-06-02 15:15:06 +00:00
LucainandClaude Opus 4.6 11dfa3169d Add vLLM library info to Hugging Face Hub requests (#43857) 
Signed-off-by: Wauplin <lucainp@gmail.com>
Signed-off-by: Lucain Pouget <lucain@huggingface.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-05-29 14:04:58 +00:00
Julien Denize 02606b0b09 [BUGFIX] Multimodal benchmark with MistralTokenizer (#42965) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
 2026-05-28 05:36:24 -07:00
Animesh Trivedi bfb9ebc211 [Feature] Add support for timed trace replay in vllm bench serve to replay Moonshot and Alibaba workload traces (#39795) 
Signed-off-by: Animesh Trivedi <Animesh.Trivedi@ibm.com>
 2026-05-28 03:31:34 -07:00
Thibault Castells 5d09f471f4 [Misc] Support interleaved custom image benchmark datasets (#43636) 
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
 2026-05-26 03:37:25 -07:00
yzong-rh 905b97adfa [Benchmark] Add num-warmup to vllm bench throughput (#43245) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-05-21 05:13:15 +00:00
pmaybankgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fadf5d332c add enqueue all option to throughput benchmark (#42975) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Signed-off-by: pmaybank <113125070+pmaybank@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 20:16:02 -07:00
Sergei SkvortsovandMichael Goin 9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-15 13:01:21 -07:00
Yasmin Moslemandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 28ee78af54 Implement custom dataset class for ASR benchmarking (#41576) 
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 12:17:58 +08:00
noobHappylife 06a60d3dd0 Fix spec decode benchmark metrics (#41916) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
 2026-05-07 09:23:21 -07:00
Akim Tsvigun 894a02500b [Bench] Forward --seed to CustomDataset and CustomMMDataset shuffle (#40788) 
Signed-off-by: akimtsvigun <akimtsvigun@gmail.com>
 2026-05-04 00:39:10 +00:00
pmaybankandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 5b39b268f5 hf_name argument for vllm bench throughput CLI (#41012) 
Signed-off-by: Philip Maybank <pmaybank@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 12:57:58 +00:00
zhrrr ea74f701db Bugfix: fix SpecBench sample argument error (#40927) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2026-04-28 00:33:49 -07:00
Sophie du Couédicandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ff2c2bd80a [Docs]Add documentation for bench serve visualization arguments (#40539) 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-23 15:48:29 -07:00
Talor Abramovichandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 7b1e0b07d0 [Bugfix] Fix dataset name and path argument validation bug in vllm bench serve (#40288) 
Signed-off-by: talora <talora@nvidia.com>
Signed-off-by: Talor Abramovich <talor19@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-21 06:14:28 -07:00
Michael Goin 1948d0c467 [UX] Defer some imports on CLI paths to save ~2s (#40056) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-16 19:48:37 -07:00
Yufeng He de111f3246 [Bugfix] Fix bench_serve UTF-8 decode crash on split multi-byte chars (#38732) 2026-04-16 12:01:25 -07:00
Talor AbramovichandBenjamin Chislett 3cc328a4be [SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029) 
Signed-off-by: talora <talora@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-04-15 12:00:07 -04:00
Lucas Kabela c08f3b2a62 Measure encoder compile time seperate from llm backbone (#39240) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-14 10:52:49 -04:00
Matthias Gehre 1e4748c66a [Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" (#38654) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-04-14 11:00:40 +00:00
Julien Debache 893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-04-14 09:20:03 +00:00
nemanjaudovic 1a2c17634e [Bugfix] Add missing ASRDataset import and CLI args in benchmarks/throughput.py (#38114) 
Signed-off-by: nemanjaudovic <nudovic@amd.com>
 2026-04-08 13:53:53 +00:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Harry Mellor d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
Isotr0py c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
Flora Feng 9040151fe1 [V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-20 11:31:43 +08:00