iHore/vLLM-4x-RTX-3090
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6 Commits 1 Branch 0 Tags
27f50debe0eea590d4aee2b07327873eb392071e
Commit Graph
6 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MOVZXandHinata 27f50debe0 feat: Add benchmark wrapper script 
Wrap llama-benchy for this box. Take prompt length, generation length,
concurrency, and context depth as arguments, and pass the rest to
llama-benchy. Defaults: pp=32768, tg=2048, concurrency 1, depth 0.

Validate the arguments, report a missing llama-benchy, and allow
BASE_URL, MODEL, and TOKENIZER overrides from the environment.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 07:39:39 +00:00
MOVZXandHinata d365f4586f docs: Add power-limit benchmark results 
Record the 220 W, 250 W, and 285 W sweep on PCIe Gen4 x16 and Gen3 x16.
State the testbed, the reproduce commands, and the results. Add the
findings: prefill scales sub-linearly, +12.9% from 220 W to 285 W on
Gen4, while decode stays inside the measured spread. Recommend Gen4 at
250 W.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 07:26:46 +00:00
MOVZXandHinata 5cc9ee2505 chore: Reset vLLM logging interval back to default 
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:37:23 +00:00
MOVZXandHinata 7e80cc67ee feat: Add pretty vLLM logging scripts 
```
[10-15 06:31:36] PP:    0.0 t/s | TG:    4.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  11.1%
[10-15 06:31:37] PP:    0.0 t/s | TG:    5.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  50.0%
[10-15 06:31:38] PP:    0.0 t/s | TG:    6.0 t/s | K/V Cache:  60.7% | Pfx Cache:   0.0% | Draft Acc:  33.3%
[10-15 06:31:39] PP:    0.0 t/s | TG:   10.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  77.8%
[10-15 06:31:40] PP:    0.0 t/s | TG:    6.0 t/s | K/V Cache:  63.2% | Pfx Cache:   0.0% | Draft Acc:  66.7%
[10-15 06:31:41] PP:    0.0 t/s | TG:    5.0 t/s | K/V Cache:  62.3% | Pfx Cache:   0.0% | Draft Acc:  22.2%
[10-15 06:31:42] PP:    0.0 t/s | TG:    8.0 t/s | K/V Cache:  63.2% | Pfx Cache:   0.0% | Draft Acc: 100.0%
[10-15 06:31:43] PP:    0.0 t/s | TG:   11.0 t/s | K/V Cache:  64.8% | Pfx Cache:   0.0% | Draft Acc:  88.9%
```

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:36:24 +00:00
MOVZX eb8c302bb3 fix: Adjust start script to avoid crashing on high pressure KV cache 
Stress-tested using 4 concurrent requests with 240K tokens each:
```
(APIServer pid=1) INFO 10-15 06:21:25 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 98.8%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:26 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 17.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 99.2%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:27 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:28 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.9 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 0.0%
```
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:24:25 +00:00
MOVZX 777e910dd0 Initial commit 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 01:01:52 +07:00