Volatility Optimization Is Actually Bayesian Inference

This episode explores Kohei Honda's tutorial and survey "Model Predictive Control via Probabilistic Inference," which unifies two decades of scattered research—path integral control, reinforcement learning theory, and variational inference—into a single coherent framework called PI-MPC. The discussion traces why classical gradient- and Hessian-based MPC solvers break down on contact-rich robotics, learned neural dynamics, or discontinuous costs, and why the resulting fallback to naive random-shooting sampling collapses under the curse of dimensionality. The core argument is that reframing sampling-based MPC as inference over a distribution of good control sequences—rather than search for a single optimum—yields dramatic gains in sample efficiency and parallelizability, with MPPI's Boltzmann-weighted, temperature-controlled posterior serving as the paper's central worked example. Along the way, the hosts debate whether "inference" is meaningfully different from optimization, tracing how entropy terms in algorithms like Soft Actor-Critic emerge naturally from the probabilistic framing rather than being added as an exploration hack. Listeners interested in robotics, control theory, or the mathematical bridges between classical control and modern probabilistic ML will find the episode's account of why this synthesis only became practical with GPU-scale parallel rollouts particularly compelling.

Interactive Visualization: Volatility Optimization Is Actually Bayesian Inference

Sources:
1. Model Predictive Control via Probabilistic Inference: A Tutorial and Survey — Kohei Honda, 2025
http://arxiv.org/abs/2511.08019v4
2. Constrained Model Predictive Control: Stability and Optimality — D. Q. Mayne, J. B. Rawlings, C. V. Rao, P. O. M. Scokaert, 2000
https://scholar.google.com/scholar?q=Constrained+Model+Predictive+Control%3A+Stability+and+Optimality
3. A Survey of Industrial Model Predictive Control Technology — S. Joe Qin, Thomas A. Badgwell, 2003
https://scholar.google.com/scholar?q=A+Survey+of+Industrial+Model+Predictive+Control+Technology
4. Model Predictive Control: Theory and Practice — A Survey — Carlos E. Garcia, David M. Prett, Manfred Morari, 1989
https://scholar.google.com/scholar?q=Model+Predictive+Control%3A+Theory+and+Practice+%E2%80%94+A+Survey
5. Model Predictive Path Integral Control using Covariance Variable Importance Sampling — Grady Williams, Andrew Aldrich, Evangelos A. Theodorou, 2015
https://scholar.google.com/scholar?q=Model+Predictive+Path+Integral+Control+using+Covariance+Variable+Importance+Sampling
6. Information Theoretic MPC for Model-Based Reinforcement Learning — Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M. Rehg, Byron Boots, Evangelos A. Theodorou, 2017
https://scholar.google.com/scholar?q=Information+Theoretic+MPC+for+Model-Based+Reinforcement+Learning
7. Robust Sampling Based Model Predictive Control with Sparse Objective Information — Grady Williams, Brian Goldfain, Paul Drews, Kamil Saigol, James M. Rehg, Evangelos A. Theodorou, 2018
https://scholar.google.com/scholar?q=Robust+Sampling+Based+Model+Predictive+Control+with+Sparse+Objective+Information
8. Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review — Sergey Levine, 2018
https://scholar.google.com/scholar?q=Reinforcement+Learning+and+Control+as+Probabilistic+Inference%3A+Tutorial+and+Review
9. Robot Trajectory Optimization using Approximate Inference — Marc Toussaint, 2009
https://scholar.google.com/scholar?q=Robot+Trajectory+Optimization+using+Approximate+Inference
10. Optimal Control as a Graphical Model Inference Problem — Hilbert J. Kappen, Vicenç Gómez, Manfred Opper, 2012
https://scholar.google.com/scholar?q=Optimal+Control+as+a+Graphical+Model+Inference+Problem
11. Variational Inference: A Review for Statisticians — David M. Blei, Alp Kucukelbir, Jon D. McAuliffe, 2017
https://scholar.google.com/scholar?q=Variational+Inference%3A+A+Review+for+Statisticians
12. Auto-Encoding Variational Bayes — Diederik P. Kingma, Max Welling, 2013
https://scholar.google.com/scholar?q=Auto-Encoding+Variational+Bayes
13. An Introduction to Variational Methods for Graphical Models — Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, Lawrence K. Saul, 1999
https://scholar.google.com/scholar?q=An+Introduction+to+Variational+Methods+for+Graphical+Models
14. Predictive Sampling: Real-Time Behaviour Synthesis with MuJoCo — Taylor Howell, Nimrod Gileadi, Saran Tunyasuvunakool, Kevin Zakka, Tom Erez, Yuval Tassa, 2022
https://scholar.google.com/scholar?q=Predictive+Sampling%3A+Real-Time+Behaviour+Synthesis+with+MuJoCo
15. STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation — Mohak Bhardwaj, Balakumar Sundaralingam, Arsalan Mousavian, Nathan D. Ratliff, Dieter Fox, Fabio Ramos, Byron Boots, 2021
https://scholar.google.com/scholar?q=STORM%3A+An+Integrated+Framework+for+Fast+Joint-Space+Model-Predictive+Control+for+Reactive+Manipulation
16. Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving — Grady Williams, Paul Drews, Brian Goldfain, James M. Rehg, Evangelos A. Theodorou, 2018
https://scholar.google.com/scholar?q=Information-Theoretic+Model+Predictive+Control%3A+Theory+and+Applications+to+Autonomous+Driving
17. Model-Based Diffusion for Trajectory Optimization — Chaoyi Pan, Zeji Yi, Guanya Shi, Guannan Qu, 2024
https://scholar.google.com/scholar?q=Model-Based+Diffusion+for+Trajectory+Optimization
18. TD-MPC2: Scalable, Robust World Models for Continuous Control — Nicklas Hansen, Hao Su, Xiaolong Wang, 2023
https://scholar.google.com/scholar?q=TD-MPC2%3A+Scalable%2C+Robust+World+Models+for+Continuous+Control
19. Recent Advances in Path Integral Control for Trajectory Optimization: An Overview in Theoretical and Algorithmic Perspectives — Muhammad Kazim, Jungee Hong, Min-Gyeom Kim, Kwang-Ki K. Kim, 2024
https://scholar.google.com/scholar?q=Recent+Advances+in+Path+Integral+Control+for+Trajectory+Optimization%3A+An+Overview+in+Theoretical+and+Algorithmic+Perspectives
Interactive Visualization: Volatility Optimization Is Actually Bayesian Inference