iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
181 Commits
Author SHA1 Message Date
Roger Wang 43c146ca42 [Misc] Clean up unnecessary E501 ignore (#26274) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-10-06 07:29:18 +00:00
Yasmin Moslem 7c2ec0fe87 [Benchmarking] Add disable_shuffle option for dataset loading (#26258) 
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
 2025-10-06 07:05:44 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Sergei SkvortsovandSergei Skvortsov b71fcd4905 [Misc] Add penalties sampling parameters to serve tool (#25974) 
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
 2025-10-03 15:43:14 -07:00
Ekagra RanjanandRoger Wang ad2d788016 [Bug][Benchmark] Fix duplicate req in oversampling (#26140) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-03 02:55:24 +00:00
Ekagra Ranjan 1cab2f9cad EAGLE 3: Fix preamble so that measured speedup over Eagle 1 becomes 32% instead of 5% on MTBench (#25916) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-10-02 11:29:35 -07:00
Nathan Scott f9e714813a [Benchmark] Finish documented v0.11.0 deprecation of --endpoint-type (#26007) 
Signed-off-by: Nathan Scott <nathans@redhat.com>
 2025-10-01 12:41:57 +00:00
Zhuohan Li d3bd171123 [Benchmark] Support benchmark throughput for external launcher DP (#25913) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-30 01:43:57 +00:00
weiliangandRoger Wang f4e4088c99 Fix random dataset mismatched token length with config. (#24937) 
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-28 08:23:44 +00:00
WeiQing ChenJunhongLJH-LBJ
 		f1d53d150c [Multimodal][Speculative Decoding]Eagle Eagle3 mm support, enablement on qwen2.5vl (#22872) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: LJH-LBJ <98734602+LJH-LBJ@users.noreply.github.com>
 2025-09-27 03:35:47 +00:00
Lucia FangandLucia Fang eea1783989 [benchmarks]allow skip ready check for bench serve (#25420) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-09-23 03:21:48 +00:00
samzong ce75e15373 refactor(benchmarks): add type annotations to wait_for_endpoint parameters (#25218) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 16:36:52 +00:00
Roger Wang 21da73343a [Misc] Clean up flags in vllm bench serve (#25138) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-18 12:43:33 +00:00
PunitvaraandHarry Mellor 05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
Simon Mo a904ea78ea [benchmark] add peak throughput metrics and plot (#23867) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-09-17 22:30:02 -07:00
samzong 4a2d33e371 [Docs] vllm/benchmarks/datasets.py fix docstring param format. (#24970) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-17 08:11:51 -07:00
samzong 47f670b03b [Docs] improve code formatting and comments for eliminate griffe build warning. (#25010) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-17 07:31:20 -07:00
Zhuohan Li 6c47f6bfa4 [Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Isotr0py 5a411ef6c4 [Benchmarks] Add MMVU video dataset support and clean up deprecated datasets (#24719) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-17 03:29:43 +00:00
Ye (Charlotte) Qi ff68035932 [Benchmarks] Throw usage error when using dataset-name random and dataset-path together (#24819) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-14 17:50:01 +00:00
Clayton Coleman bc636f21a6 [Benchmark] Allow arbitrary headers to be passed to benchmarked endpoints (#23937) 
Signed-off-by: Clayton Coleman <smarterclayton@gmail.com>
 2025-09-12 13:57:53 -07:00
Didier Durand bcb06d7baf [Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
Tomas Ruiz ee0bc5e1b4 Enable --profile in 'vllm bench throughput' (#24575) 
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
 2025-09-10 23:06:19 -07:00
Ekagra Ranjan fb1a8f932a [Benchmark] Add option to skip oversampling in benchmark (#24457) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-09-09 22:00:17 +00:00
Ming Yang 1823a00d67 [Misc] Support bench serve long context (#24373) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-09-08 22:53:10 -07:00
Ekagra RanjanandRoger Wang 41183c1fe0 [Spec Decode] Fix offline spec_decode.py (#24257) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-08 20:44:13 +00:00
Ekagra RanjanandRoger Wang cd08636926 [Spec Decode][Benchmark] Add Blitzedit dataset (#23605) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-08 10:32:52 -07:00
Ekagra Ranjan 3feeeb9fea [Spec Decode][Benchmark] Add Spec Bench Dataset for benchmarking (#23563) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2025-09-08 10:32:42 -07:00
co63oc 1bd007f234 fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
Jiangyun Zhu c83c4ff815 [Benchmark] Add support for local hf dataset path in benchmark (#23999) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-09-02 17:49:16 +00:00
Huy Do 6ace2f72b0 Fix writing benchmark results with tuple keys (#23633) 
Signed-off-by: Huy Do <huydhn@gmail.com>
 2025-08-26 19:16:09 +08:00
Jiangyun Zhu 3ecbb14b81 [Benchmarks] add benchmark for embedding models (#23000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-08-25 23:57:08 -07:00
Breno Baldas Skuk 0cb7b065c3 Feature/benchmark/random mm data/images (#23119) 
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
 2025-08-25 01:28:35 -07:00
Jared O'ConnellandCyrus Leung 31282401b6 [BugFix] Fix Python 3.9 Support (#23306) 
Signed-off-by: Jared O'Connell <46976761+jaredoconnell@users.noreply.github.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-20 23:23:56 -07:00
Cyrus Leung 0c31e28e95 [Bugfix] Fix extra whitespace in strings caused by newline (#23272) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-20 22:03:00 -07:00
Zhewen LiandSimon Mo f729023272 [CI/Build] Also check DP in benchmarks throughput script (#23038) 
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-08-20 04:09:27 +00:00
Chenheli Hua 1630cc8d0f [Benchmarks] Add video inputs to ShareGPTDataset. (#23199) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-08-19 23:42:31 +00:00
Ruixiang Tan 03d4235fd2 [Misc] Fix the benchmark's README and improve the error messages for the benchmark's argument checks (#22654) 
Signed-off-by: tanruixiang <tanruixiang0104@gmail.com>
 2025-08-19 10:18:51 -07:00
hustxiayang 31436e8b4f [Misc] Add request_id into benchmark_serve.py (#23065) 
Signed-off-by: yangxia <yangxiast@gmail.com>
 2025-08-19 08:32:18 +00:00
Seiji EicherandRoger Wang de9cb61763 Add docs for PrefixRepetitionDataset + enable usage with vllm bench throughput (#23012) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-08-16 10:21:20 +00:00
Seiji Eicher 00d6cba0cf Add PrefixRepetitionRandomDataset to vllm bench serve datasets (#20638) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-08-15 14:09:23 -07:00
Chenheli Hua 993d3d122b [Benchmarks] Include image data when ShareGPT4V dataset is used. (#22955) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-08-15 18:23:06 +00:00
Harry Mellor bc1d02ac85 [Docs] Add comprehensive CLI reference for all large vllm subcommands (#22601) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-11 00:13:33 -07:00
Breno Baldas Skuk 65a7917be4 Fix(benchmarks): allow multiple mm contents in OpenAI Chat Completion Benchmarks (#22534) 
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
 2025-08-10 09:03:15 -07:00
lkchen 808a7b69df [bench] Fix benchmark/serve.py to ignore unavailable results (#22382) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-08-07 23:15:50 -07:00
lkchen 4d4297e8fe [Bench] Split serve.py:main into async/async versions (#22405) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-08-06 23:05:07 -07:00
Lionel Villard ad6c655dde preload heavy modules when mp method is forkserver (#22214) 
Signed-off-by: Lionel Villard <villard@us.ibm.com>
 2025-08-06 20:33:24 -07:00
Seiji Eicher 6f5478298d Use aiohttp connection pool for benchmarking (#21981) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-08-03 19:23:32 -07:00
Ye (Charlotte) QiandRoger Wang 3f36c325fa [Benchmark] Support ready check timeout in vllm bench serve (#21696) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-08-03 00:52:38 -07:00
Peter Pan 533db0935d [benchmark] add max-concurrency in result table (#21095) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-30 01:15:43 -07:00