Files

T

MOVZXandHinata d365f4586f docs: Add power-limit benchmark results

Record the 220 W, 250 W, and 285 W sweep on PCIe Gen4 x16 and Gen3 x16.
State the testbed, the reproduce commands, and the results. Add the
findings: prefill scales sub-linearly, +12.9% from 220 W to 285 W on
Gen4, while decode stays inside the measured spread. Recommend Gen4 at
250 W.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>

2026-10-15 07:26:46 +00:00

3.0 KiB

Raw Blame History

GPU Power Limit Effects

Power-limit sweep on the 4x RTX 3090 box: 220 W, 250 W, 285 W, on PCIe Gen4 x16 and Gen3 x16.

Testbed

Item	Value
CPU	AMD EPYC 7642 (48c/96t)
RAM	8-Channel DDR4 2999 512 GB
GPUs	4 x RTX 3090 24 GB GDDR6X
Driver	615.71.09
Engine	vLLM 0.30.0 + `patches/`, tree head `60bdc49`
Model	Swift-1.5-Qwen3.8-Flash-Next-W4A16, TP2 x PP2 + EP, MTP K=3
Config	shipped `start.sh` profile, concurrency 1

Reproduce

# Prefill
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
  --latency-mode api --tokenizer <checkpoint-dir> \
  --pp 32768 --tg 1 --no-cache --exact-tg --concurrency 1

# Decode
llama-benchy --base-url http://127.0.0.1:8080/v1 --model Hinata \
  --latency-mode api --tokenizer <checkpoint-dir> \
  --pp 1 --tg 4096 --no-cache --exact-tg --concurrency 1

Results

PCIe Gen4 x16

Limit	Prefill t/s	TTFT ms	Decode t/s	Decode peak
220 W	5547.02 ± 8.25	5911.18 ± 9.08	172.00 ± 15.35	202.00 ± 9.42
250 W	5997.96 ± 23.68	5466.89 ± 21.61	185.00 ± 12.86	211.67 ± 5.56
285 W	6265.40 ± 32.12	5233.77 ± 26.88	176.22 ± 19.21	207.00 ± 7.48

PCIe Gen3 x16

Limit	Prefill t/s	TTFT ms	Decode t/s	Decode peak
220 W	4980.09 ± 12.33	6583.13 ± 16.30	163.10 ± 4.65	197.00 ± 5.72
250 W	5215.56 ± 10.92	6286.05 ± 13.08	157.53 ± 8.35	191.33 ± 8.18
285 W	5400.92 ± 19.10	6070.85 ± 21.66	174.04 ± 17.05	196.67 ± 10.50

Findings

Prefill scales sub-linearly with the limit. 220 → 285 W: +12.9% (Gen4), +8.4% (Gen3). 220 → 250 W gives +8.1% / +4.7%; 250 → 285 W gives only +4.5% / +3.6%.
Prefill t/s per watt (Gen4): 25.2, 24.0, 22.0. Peak efficiency at 220 W, peak throughput at 285 W.
Gen4 over Gen3: +11.4%, +15.0%, +16.0% at 220/250/285 W. Marginal watts are worth more on the wider link.
Decode does not respond to the limit. It is memory-bandwidth bound. Every decode delta fits inside the spread of the cells it joins; the 250 W extremes are noise.
Noise floor: prefill spread < 0.6% of mean, decode spread up to 11%. Meaningful deltas: > 1% on prefill, > 10% on decode.

Recommendation

Run Gen4 at 250 W. +8.1% prefill over 220 W at 95% of its t/s/W; 285 W adds +4.5% prefill for 14% more power. The power limit is not the lever for decode.

Test date: 2026-10-15

3.0 KiB Raw Blame History