diff --git a/benchmarks/gpu-power-limit-effects-en.md b/benchmarks/gpu-power-limit-effects-en.md new file mode 100644 index 0000000..9b4939b --- /dev/null +++ b/benchmarks/gpu-power-limit-effects-en.md @@ -0,0 +1,72 @@ +# GPU Power Limit Effects + +Power-limit sweep on the 4x RTX 3090 box: 220 W, 250 W, 285 W, on PCIe Gen4 x16 +and Gen3 x16. + +## Testbed + +| Item | Value | +| ------ | ----------------------------------------------------------- | +| CPU | AMD EPYC 7642 (48c/96t) | +| RAM | 8-Channel DDR4 2999 512 GB | +| GPUs | 4 x RTX 3090 24 GB GDDR6X | +| Driver | 615.71.09 | +| Engine | vLLM 0.30.0 + `patches/`, tree head `60bdc49` | +| Model | Swift-1.5-Qwen3.8-Flash-Next-W4A16, TP2 x PP2 + EP, MTP K=3 | +| Config | shipped `start.sh` profile, concurrency 1 | + +## Reproduce + +```bash +# Prefill +llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \ + --latency-mode api --tokenizer \ + --pp 32768 --tg 1 --no-cache --exact-tg --concurrency 1 + +# Decode +llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \ + --latency-mode api --tokenizer \ + --pp 1 --tg 4096 --no-cache --exact-tg --concurrency 1 +``` + +## Results + +### PCIe Gen4 x16 + +| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak | +| ----: | ------------------: | --------------: | -----------------: | ------------: | +| 220 W | **5547.02** ± 8.25 | 5911.18 ± 9.08 | **172.00** ± 15.35 | 202.00 ± 9.42 | +| 250 W | **5997.96** ± 23.68 | 5466.89 ± 21.61 | **185.00** ± 12.86 | 211.67 ± 5.56 | +| 285 W | **6265.40** ± 32.12 | 5233.77 ± 26.88 | **176.22** ± 19.21 | 207.00 ± 7.48 | + +### PCIe Gen3 x16 + +| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak | +| ----: | ------------------: | --------------: | -----------------: | -------------: | +| 220 W | **4980.09** ± 12.33 | 6583.13 ± 16.30 | **163.10** ± 4.65 | 197.00 ± 5.72 | +| 250 W | **5215.56** ± 10.92 | 6286.05 ± 13.08 | **157.53** ± 8.35 | 191.33 ± 8.18 | +| 285 W | **5400.92** ± 19.10 | 6070.85 ± 21.66 | **174.04** ± 17.05 | 196.67 ± 10.50 | + +## Findings + +1. Prefill scales sub-linearly with the limit. 220 → 285 W: +12.9% (Gen4), + +8.4% (Gen3). 220 → 250 W gives +8.1% / +4.7%; 250 → 285 W gives only + +4.5% / +3.6%. +2. Prefill t/s per watt (Gen4): 25.2, 24.0, 22.0. Peak efficiency at 220 W, + peak throughput at 285 W. +3. Gen4 over Gen3: +11.4%, +15.0%, +16.0% at 220/250/285 W. Marginal watts are + worth more on the wider link. +4. Decode does not respond to the limit. It is memory-bandwidth bound. Every + decode delta fits inside the spread of the cells it joins; the 250 W + extremes are noise. +5. Noise floor: prefill spread < 0.6% of mean, decode spread up to 11%. + Meaningful deltas: > 1% on prefill, > 10% on decode. + +## Recommendation + +Run Gen4 at 250 W. +8.1% prefill over 220 W at 95% of its t/s/W; 285 W adds ++4.5% prefill for 14% more power. The power limit is not the lever for decode. + +--- + +Test date: 2026-10-15 diff --git a/benchmarks/gpu-power-limit-effects-id.md b/benchmarks/gpu-power-limit-effects-id.md new file mode 100644 index 0000000..2fb194e --- /dev/null +++ b/benchmarks/gpu-power-limit-effects-id.md @@ -0,0 +1,73 @@ +# Efek Power Limit GPU + +Uji ini mengukur power limit pada mesin 4x RTX 3090: 220 W, 250 W, dan 285 W. +Uji ini memakai PCIe Gen4 x16 dan Gen3 x16. + +## Platform Uji + +| Item | Nilai | +| ------ | ----------------------------------------------------------- | +| CPU | AMD EPYC 7642 (48c/96t) | +| RAM | 8-Channel DDR4 2999 512 GB | +| GPUs | 4 x RTX 3090 24 GB GDDR6X | +| Driver | 615.71.09 | +| Engine | vLLM 0.30.0 + `patches/`, tree head `60bdc49` | +| Model | Swift-1.5-Qwen3.8-Flash-Next-W4A16, TP2 x PP2 + EP, MTP K=3 | +| Config | profil bawaan `start.sh`, concurrency 1 | + +## Reproduksi + +```bash +# Prefill +llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \ + --latency-mode api --tokenizer \ + --pp 32768 --tg 1 --no-cache --exact-tg --concurrency 1 + +# Decode +llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \ + --latency-mode api --tokenizer \ + --pp 1 --tg 4096 --no-cache --exact-tg --concurrency 1 +``` + +## Hasil + +### PCIe Gen4 x16 + +| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak | +| ----: | ------------------: | --------------: | -----------------: | ------------: | +| 220 W | **5547.02** ± 8.25 | 5911.18 ± 9.08 | **172.00** ± 15.35 | 202.00 ± 9.42 | +| 250 W | **5997.96** ± 23.68 | 5466.89 ± 21.61 | **185.00** ± 12.86 | 211.67 ± 5.56 | +| 285 W | **6265.40** ± 32.12 | 5233.77 ± 26.88 | **176.22** ± 19.21 | 207.00 ± 7.48 | + +### PCIe Gen3 x16 + +| Limit | Prefill t/s | TTFT ms | Decode t/s | Decode peak | +| ----: | ------------------: | --------------: | -----------------: | -------------: | +| 220 W | **4980.09** ± 12.33 | 6583.13 ± 16.30 | **163.10** ± 4.65 | 197.00 ± 5.72 | +| 250 W | **5215.56** ± 10.92 | 6286.05 ± 13.08 | **157.53** ± 8.35 | 191.33 ± 8.18 | +| 285 W | **5400.92** ± 19.10 | 6070.85 ± 21.66 | **174.04** ± 17.05 | 196.67 ± 10.50 | + +## Temuan + +1. Prefill bertambah secara sub-linier terhadap batas daya. 220 → 285 W: + +12,9% (Gen4), +8,4% (Gen3). 220 → 250 W memberi +8,1% / +4,7%. + 250 → 285 W hanya memberi +4,5% / +3,6%. +2. Nilai prefill per watt (Gen4): 25,2, lalu 24,0, lalu 22,0 t/s. Efisiensi + tertinggi ada di 220 W. Throughput tertinggi ada di 285 W. +3. Gen4 di atas Gen3: +11,4%, +15,0%, +16,0% pada 220/250/285 W. Tambahan watt + bernilai lebih besar pada link yang lebih lebar. +4. Decode tidak merespons batas daya. Decode dibatasi bandwidth memori. Selisih + decode selalu lebih kecil daripada sebaran kedua sel. Nilai ekstrem pada + 250 W hanyalah derau. +5. Batas derau: sebaran prefill < 0,6% dari rata-rata, sebaran decode sampai + 11%. Selisih yang berarti: > 1% untuk prefill, > 10% untuk decode. + +## Rekomendasi + +Pakai Gen4 pada 250 W. Batas itu memberi prefill +8,1% di atas 220 W dengan +95% t/s/W dari 220 W. 285 W menambah prefill +4,5% dengan daya 14% lebih +besar. Batas daya bukan penentu untuk decode. + +--- + +Tanggal uji: 2026-10-15