AI R&D Automation and the Intelligence Explosion Question

"What if automating AI R&D triggers an intelligence explosion?" — Alan Chan, Sören Mindermann et al. (incl. Hinton, Bengio), 2026 · policy analysis with a short quantitative supplement
arXiv 2609.36054

Is automation already happening?Reported

Anthropic figures cited by the paper. Toggle the metric.

Task-length horizon doublingIllustrative

METR time horizon (50% reliability) doubles roughly every 3 months. Curve shows the shape only: relative multiple, log axis.

The ladder: climbed, not finished

Caveats the hosts stress: coding benchmarks ≠ research ability.

The two-part feedback loop

Click a node. Software only: a better model re-enters R&D almost immediately; chips need fabs.
Click a node.

Model: dA/dt ∝ A1−β · Eλ, with E = kA

Substituting gives dA/dt = c·Aλ−β+1. Slide λ−β with β = 1 so r = λ/β = 1 + (λ−β).

Regime map: λ vs βComputed r = λ/β

Hover a cell. Outlined cell: Ho & Whitfill averaged λ = 1.40, β = 1.01 (r ≈ 1.39 at grid resolution shown as 1.4/1.0).
Hover a cell.

"Tenfold in about 17 months": doubling-by-doubling

Each doubling of A multiplies the growth rate by 2λ−β; each doubling takes 2−(λ−β) as long as the last. First doubling 4.5 months. Paper: λ−β = 0.39. Slide to test fragility (the hosts' own arithmetic uses 0.1).

Millions of researcher-equivalentsReported inputs

≈1013 tokens/day inference ÷ ≈5×105 tokens per researcher-day (RE-Bench) ≈ 2×107. Stated range 2M–200M vs thousands at a frontier company.

90% intervals for r in three subfieldsReported

Only the third clears 1, barely. No probability of r < 1 is given; point estimates are not shown here.

Which way do the known biases push r?

Per the transcript. The hosts disagree on the net direction; no sizes are given.

Estimate vs mechanism mismatch

Four frictions that could stop the loop

Click a friction for the evidence status discussed in the episode.
Click a friction.

Stakes: three channels

Policy ladder

Visibility pays off whether or not r > 1 (the hosts' reading); pacing is costlier.

References

  1. Chan, Winter, Barto, Pachocki, Hinton, Horvitz, Bengio et al. — What if automating AI R&D triggers an intelligence explosion? 2026 — arXiv 2609.36054
  2. I.J. Good — Speculations Concerning the First Ultraintelligent Machine, 1965
  3. Yudkowsky — Intelligence Explosion Microeconomics, 2013
  4. Bloom, Jones, Van Reenen, Webb — Are Ideas Getting Harder to Find? 2020
  5. Ho et al. (Epoch AI) — Algorithmic progress in language models, 2024
  6. Wijk et al. (METR) — RE-Bench, 2024
  7. Kwa, West, Becker et al. (METR) — Measuring AI Ability to Complete Long Tasks, 2025
  8. Lu et al. — The AI Scientist, 2024
  9. Kokotajlo et al. — AI 2027, 2025
  10. Ngo, Chan, Mindermann — The Alignment Problem from a Deep Learning Perspective, 2022
  11. Greenblatt et al. — AI Control, 2023
  12. Greenblatt et al. — Alignment faking in large language models, 2024
  13. Bengio, Hinton et al. — Managing extreme AI risks amid rapid progress, 2024
  14. Whitfill, Wu — Will compute bottlenecks prevent an intelligence explosion? 2025
  15. Ho, Whitfill — The software intelligence explosion debate needs experiments, 2025
  16. Eth, Davidson — Will AI R&D Automation Cause a Software Intelligence Explosion? 2025
  17. METR — Time Horizon 1.1, 2025/2026
  18. Davidson, Halperin, Houlden, Korinek — Feedback loops in innovation networks, 2026
  19. Cunningham et al. — The economics of recursive self-improvement, 2026
  20. Trammell — The bounded parallelizability of R&D, 2026
  21. Gundlach et al. — On the origin of algorithmic progress in AI, 2025
  22. Ord — The dynamics of intelligence explosions, 2026
  23. Rank et al. — PostTrainBench, 2026
  24. Lu, Lu, Lange, Clune et al. — Towards end-to-end automation of AI research, 2026
  25. Brundage et al. — Frontier AI auditing, 2026