AI Post Transformers · episode companion

Formalizing Hopfield Networks and Boltzmann Machines in Lean 4

Matteo Cipollina (Axiomatic AI), Michail Karatarakis, Freek Wiedijk (Radboud) · 2025, Sept 2026 revision · ~16,000 new Lean 4 lines on mathlib
arXiv 2512.07766 Interactive page

Proof pipeline

Click a stage.

Where the ~16,000 lines go

Figures stated in the episode.

Asynchronous descent

Stored pattern (+,+,−,+). Click a neuron to corrupt it, then update.

Schedule:

Energy trace

E = −Σi<j wij sisj for this single-pattern network.

Why equal-energy flips can't cycle

Hebbian weights W = Σ pkpkᵀ − mI

Exact arithmetic on small example sets (n=4).

Applying W to stored pattern j

W pj = (n−m) pj + cross-talk.

The paper writes out the disturbance term but states no stability theorem for non-orthogonal patterns.

Logistic update vs temperature

P(unit on) = σ(ΔE / T). As T→0 it becomes the Hopfield step rule (Lemma 4.4).

T

Stationary distribution on 3 visible units Illustrative weights

Bars: exact Boltzmann π. Dots: random-scan Gibbs counts.

Random-scan kernel P Illustrative weights

Hover a cell. Diagonal is strictly positive (aperiodic); single bit flips connect all states.

What is and isn't proved

Collatz–Wielandt function Illustrative matrix

T = [[2,1],[1,3]], x = (t, 1−t). f(x) = mini (Tx)i/xi; its maximum is the Perron root.

t

References

  1. Formalized Hopfield Networks and Boltzmann Machines — Cipollina, Karatarakis, Wiedijk, 2025
  2. A Learning Algorithm for Boltzmann Machines — Ackley, Hinton, Sejnowski, 1985
  3. Training Products of Experts by Minimizing Contrastive Divergence — Hinton, 2002
  4. A Fast Learning Algorithm for Deep Belief Nets — Hinton, Osindero, Teh, 2006
  5. Deep Boltzmann Machines — Salakhutdinov, Hinton, 2009
  6. The PageRank Citation Ranking — Page, Brin, Motwani, Winograd, 1998
  7. Markov Chains and Mixing Times — Levin, Peres, Wilmer, 2009
  8. Non-negative Matrices and Markov Chains — Seneta, 1981
  9. Hopfield Networks is All You Need — Ramsauer et al., 2020
  10. Dense Associative Memory for Pattern Recognition — Krotov, Hopfield, 2016
  11. Neural networks and physical systems with emergent collective computational abilities — Hopfield, 1982
  12. On Contrastive Divergence Learning — Carreira-Perpiñán, Hinton, 2005
  13. Stochastic Ising machines and p-bits — Camsari, Sutton, Datta, 2017-2019
  14. Formalizing Perron–Frobenius — Thiemann, Divasón et al., 2016-2019
  15. Certigrad — Selsam, Liang, Dill, 2017
  16. Attention is not all you need: rank collapse — Dong, Cordonnier, Loukas, 2021
  17. Metropolis et al. 1953 / Kirkpatrick et al. 1983