iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
53 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Woosuk KwonClaude Opus 5.5vllm-agent
 		c0b6f74bee [Benchmark] Add per-case kernel profiles to vllm bench forward (#61692) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-17 08:27:11 -07:00
znzrybandWoosuk Kwon ad2688ab30 [Bugfix] Validate draft-token capacity in forward benchmark (#61809) 
Signed-off-by: znzryb <znzryb34@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-10-17 00:51:36 -07:00
HarshilandCyrus Leung c54dca96bd [Bugfix][Benchmark] Fix probe latency and duration in bench serve (#60511) 
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-15 05:50:30 +00:00
Woosuk KwonandClaude Opus 5.5 2e8a5930f6 [Benchmark] Remove the temporary bench forward dummy loader (#61676) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 13:30:18 -07:00
Woosuk KwonandClaude Opus 5.5 9d3dfa2886 [Benchmark] Add vllm bench forward: exact-shape forward timing (#61339) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 10:52:26 -07:00
Akshan Krithick 02d69c663a [Bugfix][Benchmark] Preserve SSE output across transport chunk boundaries (#51172) 
Signed-off-by: Akshan Krithick <akshankrithick305@gmail.com>
 2026-10-13 21:58:58 -07:00
sungbin1015andFlora Feng 2ddde19e8e [Frontend] Support min_p in the Responses API (#48084) 
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-08 20:38:17 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
Prudhvi VudaandOpenAI Codex ea106db806 [Bugfix][Bench] Clean up synthetic video files and writers (#59529) 
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-02 02:51:34 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
Louie Tsai 1b77cc3d67 [Benchmark] Add sweep warmup and failure recovery (#57305) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-09-30 11:16:41 +08:00
HarshilClaudeChauncey
 		af7f9488c2 [Benchmark] Add Responses API backend to vllm bench serve (#54628) 
Signed-off-by: QHarshil <harshil_c@hotmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-28 05:04:11 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
SurajandClaude Opus 5 93e051e290 [Bugfix][Benchmark] Make streaming TTFT/E2E latency accounting consistent across endpoints (#55508) 
Signed-off-by: surajm20061998 <surajm20061998@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-13 05:20:54 +00:00
Nicolò Lucchesi 9dcab80215 [Bugfix][Bench] Fix bench mm-processor crash in shared request sampling (#56300) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-09-11 16:53:04 +08:00
Michał Ganczarenko d1922cb5a7 [Bugfix] Restore multimodal support on the plain "vllm" throughput backend (#52168) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-08-28 10:28:25 +00:00
Roy WangClaude Fable 5Bugen Zhao
 		963fcfa48c [Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test (#51592) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 08:50:51 +00:00
Jared Wen e07532b035 [MISC][Bench] refactor throughput and reuse serve's get samples (#50981) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
 2026-08-06 10:31:23 +00:00
Guan-Ming Chiu 63e78ce365 [Benchmark] Add probe requests to vllm bench serve (#49611) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-08-01 22:10:56 +08:00
mgazzandKevin H. Luu a454a1dd25 [Bugfix][Benchmarks] Restore --skip-tokenizer-init with custom dataset (#49180) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-07-24 04:30:19 -07:00
almayne d1f1d86797 [Bugfix] Re-enable benchmarking of librispeech dataset. (#47033) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
 2026-07-08 16:19:26 +00:00
Harry Mellor f1445f6dbd [CI] Bump huggingface-hub from v1.10.2 to v1.22.0 (#47551) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-04 07:45:45 -07:00
Michał Ganczarenko c8bdcc0116 [Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-07-01 15:42:27 +00:00
hurukawaOpenAI CodexRoger Wangmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		489abadfb8 feat: support to OpenMOSS-Team (#44124) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa19 <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 00:08:13 -07:00
Ekagra Ranjan 226ba9fc9e [ASR] Add Long Audio benchmark and correctness test (#44587) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-06-12 04:11:16 +00:00
LiLi ZhangClaudeChauncey
 		89c6a41001 [Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457) 
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-06-09 23:59:18 -07:00
Albert Cheng e0081ef8cf [Benchmark] Enable reasoning-model (thinking) benchmarking via --chat-template-kwargs for client-rendered datasets (#44244) 
Signed-off-by: Albert Cheng <albertching0112@gmail.com>
 2026-06-02 22:49:51 -07:00
XiaoZ 53fa09d085 [Misc] Support local image encoding in benchmarks (#43843) 
Signed-off-by: xiaoz <Sukra1@outlook.com>
 2026-06-02 15:15:06 +00:00
Thibault Castells 5d09f471f4 [Misc] Support interleaved custom image benchmark datasets (#43636) 
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
 2026-05-26 03:37:25 -07:00
Akim Tsvigun 894a02500b [Bench] Forward --seed to CustomDataset and CustomMMDataset shuffle (#40788) 
Signed-off-by: akimtsvigun <akimtsvigun@gmail.com>
 2026-05-04 00:39:10 +00:00
Julien Debache 893b2affff feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking (#30156) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-04-14 09:20:03 +00:00
Isotr0py a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Cyrus Leung d3a51da92a [Benchmark] Simplify SLA scan (#35306) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-25 22:35:41 -08:00
Fan YangandFan Yang a1946570d8 add --insecure arg to the vllm bench to skip TLS (#34026) 
Signed-off-by: Fan Yang <yan9fan@meta.com>
Co-authored-by: Fan Yang <yan9fan@meta.com>
 2026-02-10 22:23:52 +08:00
Cyrus Leung 7c0d3c5152 [Benchmark] Share data between SLA runs (#32184) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-13 01:12:22 +08:00
Cyrus Leung ef96fa3f1f [Benchmark][2/2] Use spline interpolation to tune SLA variables (#32095) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 20:27:27 -08:00
Cyrus Leung 5f2385a4c8 [Benchmark][1/2] Generalize SLA criterion validation from binary flags to margins (#32075) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 07:11:03 +00:00
Ning Xie abd9224280 resolve pydantic error in startup benchmark (#31348) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-10 02:41:27 +00:00
Michael Goin 519ef9a911 [UX] Make vllm bench serve discover model by default and use --input-len (#30816) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-17 01:55:30 -08:00
Michael Goin a450c64a30 [Bugfix] Fail instead of ignoring when CompilationConfig gets invalid args (#30708) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-15 20:18:02 +00:00
Copilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ProExpertProgLuka GovedičLuka Govedič
 		1c593e117d Fix boolean nested params, add dict format support, and enhance plotting for vllm bench sweep (#29025) 
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ProExpertProg <11367180+ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-12-02 20:40:56 +00:00
Eugene KhvedchenyaandRoger Wang 5e72216d17 Feature/video support in random mm dataset (#25963) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenia@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-29 18:24:52 +08:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Roger Wang 21da73343a [Misc] Clean up flags in vllm bench serve (#25138) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-18 12:43:33 +00:00
Ming Yang 1823a00d67 [Misc] Support bench serve long context (#24373) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-09-08 22:53:10 -07:00
Breno Baldas Skuk 0cb7b065c3 Feature/benchmark/random mm data/images (#23119) 
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
 2025-08-25 01:28:35 -07:00