docs: Add power-limit benchmark results
Record the 220 W, 250 W, and 285 W sweep on PCIe Gen4 x16 and Gen3 x16. State the testbed, the reproduce commands, and the results. Add the findings: prefill scales sub-linearly, +12.9% from 220 W to 285 W on Gen4, while decode stays inside the measured spread. Recommend Gen4 at 250 W. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -0,0 +1,72 @@
|
||||
# GPU Power Limit Effects
|
||||
|
||||
Power-limit sweep on the 4x RTX 3090 box: 220 W, 250 W, 285 W, on PCIe Gen4 x16
|
||||
and Gen3 x16.
|
||||
|
||||
## Testbed
|
||||
|
||||
| Item | Value |
|
||||
| ------ | ----------------------------------------------------------- |
|
||||
| CPU | AMD EPYC 7642 (48c/96t) |
|
||||
| RAM | 8-Channel DDR4 2999 512 GB |
|
||||
| GPUs | 4 x RTX 3090 24 GB GDDR6X |
|
||||
| Driver | 615.71.09 |
|
||||
| Engine | vLLM 0.30.0 + `patches/`, tree head `60bdc49` |
|
||||
| Model | Swift-1.5-Qwen3.8-Flash-Next-W4A16, TP2 x PP2 + EP, MTP K=3 |
|
||||
| Config | shipped `start.sh` profile, concurrency 1 |
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
# Prefill
|
||||
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
|
||||
--latency-mode api --tokenizer <checkpoint-dir> \
|
||||
--pp 32768 --tg 1 --no-cache --exact-tg --concurrency 1
|
||||
|
||||
# Decode
|
||||
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
|
||||
--latency-mode api --tokenizer <checkpoint-dir> \
|
||||
--pp 1 --tg 4096 --no-cache --exact-tg --concurrency 1
|
||||
```
|
||||
|
||||
## Results
|
||||
|
||||
### PCIe Gen4 x16
|
||||
|
||||
| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak |
|
||||
| ----: | ------------------: | --------------: | -----------------: | ------------: |
|
||||
| 220 W | **5547.02** ± 8.25 | 5911.18 ± 9.08 | **172.00** ± 15.35 | 202.00 ± 9.42 |
|
||||
| 250 W | **5997.96** ± 23.68 | 5466.89 ± 21.61 | **185.00** ± 12.86 | 211.67 ± 5.56 |
|
||||
| 285 W | **6265.40** ± 32.12 | 5233.77 ± 26.88 | **176.22** ± 19.21 | 207.00 ± 7.48 |
|
||||
|
||||
### PCIe Gen3 x16
|
||||
|
||||
| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak |
|
||||
| ----: | ------------------: | --------------: | -----------------: | -------------: |
|
||||
| 220 W | **4980.09** ± 12.33 | 6583.13 ± 16.30 | **163.10** ± 4.65 | 197.00 ± 5.72 |
|
||||
| 250 W | **5215.56** ± 10.92 | 6286.05 ± 13.08 | **157.53** ± 8.35 | 191.33 ± 8.18 |
|
||||
| 285 W | **5400.92** ± 19.10 | 6070.85 ± 21.66 | **174.04** ± 17.05 | 196.67 ± 10.50 |
|
||||
|
||||
## Findings
|
||||
|
||||
1. Prefill scales sub-linearly with the limit. 220 → 285 W: +12.9% (Gen4),
|
||||
+8.4% (Gen3). 220 → 250 W gives +8.1% / +4.7%; 250 → 285 W gives only
|
||||
+4.5% / +3.6%.
|
||||
2. Prefill t/s per watt (Gen4): 25.2, 24.0, 22.0. Peak efficiency at 220 W,
|
||||
peak throughput at 285 W.
|
||||
3. Gen4 over Gen3: +11.4%, +15.0%, +16.0% at 220/250/285 W. Marginal watts are
|
||||
worth more on the wider link.
|
||||
4. Decode does not respond to the limit. It is memory-bandwidth bound. Every
|
||||
decode delta fits inside the spread of the cells it joins; the 250 W
|
||||
extremes are noise.
|
||||
5. Noise floor: prefill spread < 0.6% of mean, decode spread up to 11%.
|
||||
Meaningful deltas: > 1% on prefill, > 10% on decode.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Run Gen4 at 250 W. +8.1% prefill over 220 W at 95% of its t/s/W; 285 W adds
|
||||
+4.5% prefill for 14% more power. The power limit is not the lever for decode.
|
||||
|
||||
---
|
||||
|
||||
Test date: 2026-10-15
|
||||
@@ -0,0 +1,73 @@
|
||||
# Efek Power Limit GPU
|
||||
|
||||
Uji ini mengukur power limit pada mesin 4x RTX 3090: 220 W, 250 W, dan 285 W.
|
||||
Uji ini memakai PCIe Gen4 x16 dan Gen3 x16.
|
||||
|
||||
## Platform Uji
|
||||
|
||||
| Item | Nilai |
|
||||
| ------ | ----------------------------------------------------------- |
|
||||
| CPU | AMD EPYC 7642 (48c/96t) |
|
||||
| RAM | 8-Channel DDR4 2999 512 GB |
|
||||
| GPUs | 4 x RTX 3090 24 GB GDDR6X |
|
||||
| Driver | 615.71.09 |
|
||||
| Engine | vLLM 0.30.0 + `patches/`, tree head `60bdc49` |
|
||||
| Model | Swift-1.5-Qwen3.8-Flash-Next-W4A16, TP2 x PP2 + EP, MTP K=3 |
|
||||
| Config | profil bawaan `start.sh`, concurrency 1 |
|
||||
|
||||
## Reproduksi
|
||||
|
||||
```bash
|
||||
# Prefill
|
||||
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
|
||||
--latency-mode api --tokenizer <checkpoint-dir> \
|
||||
--pp 32768 --tg 1 --no-cache --exact-tg --concurrency 1
|
||||
|
||||
# Decode
|
||||
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
|
||||
--latency-mode api --tokenizer <checkpoint-dir> \
|
||||
--pp 1 --tg 4096 --no-cache --exact-tg --concurrency 1
|
||||
```
|
||||
|
||||
## Hasil
|
||||
|
||||
### PCIe Gen4 x16
|
||||
|
||||
| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak |
|
||||
| ----: | ------------------: | --------------: | -----------------: | ------------: |
|
||||
| 220 W | **5547.02** ± 8.25 | 5911.18 ± 9.08 | **172.00** ± 15.35 | 202.00 ± 9.42 |
|
||||
| 250 W | **5997.96** ± 23.68 | 5466.89 ± 21.61 | **185.00** ± 12.86 | 211.67 ± 5.56 |
|
||||
| 285 W | **6265.40** ± 32.12 | 5233.77 ± 26.88 | **176.22** ± 19.21 | 207.00 ± 7.48 |
|
||||
|
||||
### PCIe Gen3 x16
|
||||
|
||||
| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak |
|
||||
| ----: | ------------------: | --------------: | -----------------: | -------------: |
|
||||
| 220 W | **4980.09** ± 12.33 | 6583.13 ± 16.30 | **163.10** ± 4.65 | 197.00 ± 5.72 |
|
||||
| 250 W | **5215.56** ± 10.92 | 6286.05 ± 13.08 | **157.53** ± 8.35 | 191.33 ± 8.18 |
|
||||
| 285 W | **5400.92** ± 19.10 | 6070.85 ± 21.66 | **174.04** ± 17.05 | 196.67 ± 10.50 |
|
||||
|
||||
## Temuan
|
||||
|
||||
1. Prefill bertambah secara sub-linier terhadap batas daya. 220 → 285 W:
|
||||
+12,9% (Gen4), +8,4% (Gen3). 220 → 250 W memberi +8,1% / +4,7%.
|
||||
250 → 285 W hanya memberi +4,5% / +3,6%.
|
||||
2. Nilai prefill per watt (Gen4): 25,2, lalu 24,0, lalu 22,0 t/s. Efisiensi
|
||||
tertinggi ada di 220 W. Throughput tertinggi ada di 285 W.
|
||||
3. Gen4 di atas Gen3: +11,4%, +15,0%, +16,0% pada 220/250/285 W. Tambahan watt
|
||||
bernilai lebih besar pada link yang lebih lebar.
|
||||
4. Decode tidak merespons batas daya. Decode dibatasi bandwidth memori. Selisih
|
||||
decode selalu lebih kecil daripada sebaran kedua sel. Nilai ekstrem pada
|
||||
250 W hanyalah derau.
|
||||
5. Batas derau: sebaran prefill < 0,6% dari rata-rata, sebaran decode sampai
|
||||
11%. Selisih yang berarti: > 1% untuk prefill, > 10% untuk decode.
|
||||
|
||||
## Rekomendasi
|
||||
|
||||
Pakai Gen4 pada 250 W. Batas itu memberi prefill +8,1% di atas 220 W dengan
|
||||
95% t/s/W dari 220 W. 285 W menambah prefill +4,5% dengan daya 14% lebih
|
||||
besar. Batas daya bukan penentu untuk decode.
|
||||
|
||||
---
|
||||
|
||||
Tanggal uji: 2026-10-15
|
||||
Reference in New Issue
Block a user