One stack, looped, read out after every loop
Drag the loop budget. The same 24 layers rerun; a typed readout gives one probability per declared option.
Loops K = 4
Compute grid: 24 layers × K loops
Schematic of work done, not measured data. Hover a cell. Same weights, repeated.Illustrative
Typed options declared by the caller
Bar heights are placeholders to show output shape: one probability per option, summing to 1.Illustrative
System 1 vs 1.5 vs 2
The Kahneman labels are the paper's framing, not evidence.
Why fixed depth hurts: liar chains (Jev API)
Reported points only: 100% at 1 step, 62.5% at 3 steps, chance from 6 steps. Dashed segments are not measured.
Lineage of looping (click a node)
Where the loss attaches
LoRA adapters on a frozen backbone
Rank 64 shared across loops, plus a small per-loop readout correction: about 60.6M shared, roughly 61M trainable.
Training recipe
Results explorer
Caveats and the RL case study
Open questions the episode raises
References
- SanSi — Gan, Lee, Kang, 2026
- Strictly Proper Scoring Rules — Gneiting, Raftery, 2007
- On Calibration of Modern Neural Networks — Guo et al., 2017
- Generative Verifiers — Zhang et al., 2024
- Universal Transformers — Dehghani et al., 2019
- Recurrent Depth / Huginn — Geiping et al., 2025
- Ouro: Scaling Latent Reasoning via Looped LMs — Zhu et al., 2025
- Adaptive Computation Time — Graves, 2016
- Multi-Scale Dense Networks — Huang et al., 2018
- PonderNet — Banino et al., 2021
- Shallow-Deep Networks — Kaya et al., 2019
- Algorithm Synthesis with Recurrent Networks — Bansal et al., 2022
- TTRL — Zuo et al., 2025
- LLM-as-a-Judge, MT-Bench — Zheng et al., 2023
- Language Models Don't Always Say What They Think — Turpin et al., 2023
- LoRA — Hu et al., 2022 (mentioned in the episode)