Intelligence per Watt: Measuring Local AI Efficiency Against the Cloud
Saad-Falcon, Narayan, et al. (Stanford, Together AI) · arXiv Nov 2025, latest version Oct 2026 · ~1M single-turn queries · 20+ models · 8 accelerators
arXiv 2511.07885
Interactive page
Measurement pipeline
As described in episode
Click a stage to see what it records.
Select a stage.
What counts as "local"?
Reported
Limit is ≤20B
active
parameters per token, not total.
Watts vs joules
Illustrative calculator
Accuracy held at 0.70 for both devices. Values are set by you, not paper data. Formulas: IPW = accuracy / power; energy = power × latency.
Four reported variants
Reported
Average vs peak power, per watt vs per joule. Hover a cell.
2023 → 2025: one best pairing per year
Table 2 as discussed
Each year changes model and chip together.
Attributed gain split
Reported
Log scale
Bars multiply. Shares depend on peel-off order; the hardware factor is a leftover ratio.
Memory bandwidth of each year's chip
Table 7
The 2025 chip has the lowest bandwidth.
Coverage
Reported
Routing simulation savings
Reported
Baseline: all queries to Qwen3-235B on one H200, batch 16. 80M queries / 24h. No real router was built.
Cloud vs local gap
Reported ratios
Log scale
All "vs M4 Max" except the batch row. Ranges are drawn as spans.
This paper vs MLPerf Power
Qualitative
Hover cells. Green = yes, red = no, amber = asserted or not covered.
Power sampling interval
Reported
Log scale
Finer sampling does not fix scope: accelerator-only, 10–15% software error.
References