iHore/vLLM-4x-RTX-3090
8
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4 Commits 1 Branch 0 Tags
5cc9ee25056ef44372a7539806ae46d499ddaa41
Commit Graph
4 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MOVZXandHinata 5cc9ee2505 chore: Reset vLLM logging interval back to default 
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:37:23 +00:00
MOVZXandHinata 7e80cc67ee feat: Add pretty vLLM logging scripts 
```
[10-15 06:31:36] PP:    0.0 t/s | TG:    4.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  11.1%
[10-15 06:31:37] PP:    0.0 t/s | TG:    5.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  50.0%
[10-15 06:31:38] PP:    0.0 t/s | TG:    6.0 t/s | K/V Cache:  60.7% | Pfx Cache:   0.0% | Draft Acc:  33.3%
[10-15 06:31:39] PP:    0.0 t/s | TG:   10.0 t/s | K/V Cache:  61.5% | Pfx Cache:   0.0% | Draft Acc:  77.8%
[10-15 06:31:40] PP:    0.0 t/s | TG:    6.0 t/s | K/V Cache:  63.2% | Pfx Cache:   0.0% | Draft Acc:  66.7%
[10-15 06:31:41] PP:    0.0 t/s | TG:    5.0 t/s | K/V Cache:  62.3% | Pfx Cache:   0.0% | Draft Acc:  22.2%
[10-15 06:31:42] PP:    0.0 t/s | TG:    8.0 t/s | K/V Cache:  63.2% | Pfx Cache:   0.0% | Draft Acc: 100.0%
[10-15 06:31:43] PP:    0.0 t/s | TG:   11.0 t/s | K/V Cache:  64.8% | Pfx Cache:   0.0% | Draft Acc:  88.9%
```

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:36:24 +00:00
MOVZX eb8c302bb3 fix: Adjust start script to avoid crashing on high pressure KV cache 
Stress-tested using 4 concurrent requests with 240K tokens each:
```
(APIServer pid=1) INFO 10-15 06:21:25 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 3.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 98.8%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:26 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 17.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 99.2%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:27 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 8.0 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 99.6%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 10-15 06:21:28 [loggers.py:315] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 20.9 tokens/s, Running: 4 reqs, Waiting: 1 reqs, GPU KV cache usage: 100.0%, Prefix cache hit rate: 0.0%
```
Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 06:24:25 +00:00
MOVZX 777e910dd0 Initial commit 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 01:01:52 +07:00