MAML and the Basics of Meta-Learning
Mar 30, 2026
This episode explores meta-learning through the lens of MAML, explaining how it differs from ordinary supervised learning and standard transfer learning by explicitly training models to adapt quickly to new tasks after just one or a few gradient updates. It walks through the core idea of optimizing for post-update performance, including the role of second-order meta-gradients and the simpler first-order approximation, while placing MAML within the broader landscape of few-shot and gradient-based meta-learning. The discussion also highlights why the paper mattered across multiple domains, covering not just classification benchmarks like Omniglot and MiniImagenet but also regression with sinusoid fitting and reinforcement learning with fast-adapting policies. A listener would find it interesting because it turns a buzzword-heavy area into a concrete framework for thinking about how models can learn to learn, setting up deeper discussions about newer systems built on these ideas.
Interactive Visualization: MAML and the Basics of Meta-Learning
Sources:
1. MAML and the Basics of Meta-Learning
https://arxiv.org/pdf/1703.03400
2.
https://par.nsf.gov/servlets/purl/10427895
https://par.nsf.gov/servlets/purl/10427895
3. Optimization as a Model for Few-Shot Learning — Sachin Ravi, Hugo Larochelle, 2017
https://scholar.google.com/scholar?q=Optimization+as+a+Model+for+Few-Shot+Learning
4. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks — Chelsea Finn, Pieter Abbeel, Sergey Levine, 2017
https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+for+Fast+Adaptation+of+Deep+Networks
5. RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning — Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel, 2016
https://scholar.google.com/scholar?q=RL%5E2%3A+Fast+Reinforcement+Learning+via+Slow+Reinforcement+Learning
6. Meta-Learning in Neural Networks: A Survey — Timothy M. Hospedales, Antreas Antoniou, Paul Micaelli, Amos J. Storkey, 2021
https://scholar.google.com/scholar?q=Meta-Learning+in+Neural+Networks%3A+A+Survey
7. Matching Networks for One Shot Learning — Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra, 2016
https://scholar.google.com/scholar?q=Matching+Networks+for+One+Shot+Learning
8. Prototypical Networks for Few-shot Learning — Jake Snell, Kevin Swersky, Richard Zemel, 2017
https://scholar.google.com/scholar?q=Prototypical+Networks+for+Few-shot+Learning
9. A Closer Look at Few-shot Classification — Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, Jia-Bin Huang, 2019
https://scholar.google.com/scholar?q=A+Closer+Look+at+Few-shot+Classification
10. Generalizing from a Few Examples — Yaqing Wang, Quanming Yao, James T. Kwok, Lionel M. Ni, 2020
https://scholar.google.com/scholar?q=Generalizing+from+a+Few+Examples
11. Meta-SGD: Learning to Learn Quickly for Few-Shot Learning — Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li, 2017
https://scholar.google.com/scholar?q=Meta-SGD%3A+Learning+to+Learn+Quickly+for+Few-Shot+Learning
12. On First-Order Meta-Learning Algorithms — Alex Nichol, Joshua Achiam, John Schulman, 2018
https://scholar.google.com/scholar?q=On+First-Order+Meta-Learning+Algorithms
13. How to Train Your MAML — Antreas Antoniou, Harrison Edwards, Amos Storkey, 2019
https://scholar.google.com/scholar?q=How+to+Train+Your+MAML
14. Meta-learning with Differentiable Closed-Form Solvers — Luca Bertinetto, Joao F. Henriques, Philip H. S. Torr, Andrea Vedaldi, 2018
https://scholar.google.com/scholar?q=Meta-learning+with+Differentiable+Closed-Form+Solvers
15. Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables — Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, Deirdre Quillen, 2019
https://scholar.google.com/scholar?q=Efficient+Off-Policy+Meta-Reinforcement+Learning+via+Probabilistic+Context+Variables
16. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning — Ronald J. Williams, 1992
https://scholar.google.com/scholar?q=Simple+Statistical+Gradient-Following+Algorithms+for+Connectionist+Reinforcement+Learning
17. Trust Region Policy Optimization — John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, Pieter Abbeel, 2015
https://scholar.google.com/scholar?q=Trust+Region+Policy+Optimization
18. Proximal Policy Optimization Algorithms — John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, 2017
https://scholar.google.com/scholar?q=Proximal+Policy+Optimization+Algorithms
19. Meta-Learning with Memory-Augmented Neural Networks — Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap, 2016
https://scholar.google.com/scholar?q=Meta-Learning+with+Memory-Augmented+Neural+Networks
20. Learning to Learn by Gradient Descent by Gradient Descent — Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Nando de Freitas, 2016
https://scholar.google.com/scholar?q=Learning+to+Learn+by+Gradient+Descent+by+Gradient+Descent
21. Transformers learn in-context by gradient descent — Johannes von Oswald et al., 2022
https://scholar.google.com/scholar?q=Transformers+learn+in-context+by+gradient+descent
22. In-context Learning and Gradient Descent Revisited — Gilad Deutch et al., 2023
https://scholar.google.com/scholar?q=In-context+Learning+and+Gradient+Descent+Revisited
23. Low-Rank Few-Shot Adaptation of Vision-Language Models — Maxime Zanella and Ismail Ben Ayed, 2024
https://scholar.google.com/scholar?q=Low-Rank+Few-Shot+Adaptation+of+Vision-Language+Models
24. Meta-Adapter: An Online Few-shot Learner for Vision-Language Model — Cheng Cheng et al., 2023
https://scholar.google.com/scholar?q=Meta-Adapter%3A+An+Online+Few-shot+Learner+for+Vision-Language+Model
25. Cross-Domain Few-Shot Learning via Adaptive Transformer Networks — Naeem Paeedeh et al., 2024
https://scholar.google.com/scholar?q=Cross-Domain+Few-Shot+Learning+via+Adaptive+Transformer+Networks
26. Few-shot Adaptation of Multi-modal Foundation Models: A Survey — Fan Liu et al., 2024
https://scholar.google.com/scholar?q=Few-shot+Adaptation+of+Multi-modal+Foundation+Models%3A+A+Survey
27. AI Post Transformers: In-Context Learning as Implicit Learning Algorithms — Hal Turing & Dr. Ada Shannon, Wed,
https://podcast.do-not-panic.com/episodes/in-context-learning-as-implicit-learning-algorithms/
28. AI Post Transformers: NVIDIA: TTT-E2E: Unlocking Long-Context Learning via End-to-End Test-Time Training — Hal Turing & Dr. Ada Shannon, Sat,
https://podcast.do-not-panic.com/episodes/nvidia-ttt-e2e-unlocking-long-context-learning-via-end-to-end-test-time-training/
29. AI Post Transformers: Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts — Hal Turing & Dr. Ada Shannon, Tue,
https://podcast.do-not-panic.com/episodes/zero-shot-context-generalization-in-reinforcement-learning-from-few-training-con/
30. AI Post Transformers: A 2024 Survey Analyzing Generalization in Deep Reinforcement Learning — Hal Turing & Dr. Ada Shannon, Fri,
https://podcast.do-not-panic.com/episodes/a-2024-survey-analyzing-generalization-in-deep-reinforcement-learning/
Interactive Visualization: MAML and the Basics of Meta-Learning
1. MAML and the Basics of Meta-Learning
https://arxiv.org/pdf/1703.03400
2.
https://par.nsf.gov/servlets/purl/10427895
https://par.nsf.gov/servlets/purl/10427895
3. Optimization as a Model for Few-Shot Learning — Sachin Ravi, Hugo Larochelle, 2017
https://scholar.google.com/scholar?q=Optimization+as+a+Model+for+Few-Shot+Learning
4. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks — Chelsea Finn, Pieter Abbeel, Sergey Levine, 2017
https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+for+Fast+Adaptation+of+Deep+Networks
5. RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning — Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel, 2016
https://scholar.google.com/scholar?q=RL%5E2%3A+Fast+Reinforcement+Learning+via+Slow+Reinforcement+Learning
6. Meta-Learning in Neural Networks: A Survey — Timothy M. Hospedales, Antreas Antoniou, Paul Micaelli, Amos J. Storkey, 2021
https://scholar.google.com/scholar?q=Meta-Learning+in+Neural+Networks%3A+A+Survey
7. Matching Networks for One Shot Learning — Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra, 2016
https://scholar.google.com/scholar?q=Matching+Networks+for+One+Shot+Learning
8. Prototypical Networks for Few-shot Learning — Jake Snell, Kevin Swersky, Richard Zemel, 2017
https://scholar.google.com/scholar?q=Prototypical+Networks+for+Few-shot+Learning
9. A Closer Look at Few-shot Classification — Wei-Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, Jia-Bin Huang, 2019
https://scholar.google.com/scholar?q=A+Closer+Look+at+Few-shot+Classification
10. Generalizing from a Few Examples — Yaqing Wang, Quanming Yao, James T. Kwok, Lionel M. Ni, 2020
https://scholar.google.com/scholar?q=Generalizing+from+a+Few+Examples
11. Meta-SGD: Learning to Learn Quickly for Few-Shot Learning — Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li, 2017
https://scholar.google.com/scholar?q=Meta-SGD%3A+Learning+to+Learn+Quickly+for+Few-Shot+Learning
12. On First-Order Meta-Learning Algorithms — Alex Nichol, Joshua Achiam, John Schulman, 2018
https://scholar.google.com/scholar?q=On+First-Order+Meta-Learning+Algorithms
13. How to Train Your MAML — Antreas Antoniou, Harrison Edwards, Amos Storkey, 2019
https://scholar.google.com/scholar?q=How+to+Train+Your+MAML
14. Meta-learning with Differentiable Closed-Form Solvers — Luca Bertinetto, Joao F. Henriques, Philip H. S. Torr, Andrea Vedaldi, 2018
https://scholar.google.com/scholar?q=Meta-learning+with+Differentiable+Closed-Form+Solvers
15. Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables — Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, Deirdre Quillen, 2019
https://scholar.google.com/scholar?q=Efficient+Off-Policy+Meta-Reinforcement+Learning+via+Probabilistic+Context+Variables
16. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning — Ronald J. Williams, 1992
https://scholar.google.com/scholar?q=Simple+Statistical+Gradient-Following+Algorithms+for+Connectionist+Reinforcement+Learning
17. Trust Region Policy Optimization — John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, Pieter Abbeel, 2015
https://scholar.google.com/scholar?q=Trust+Region+Policy+Optimization
18. Proximal Policy Optimization Algorithms — John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, 2017
https://scholar.google.com/scholar?q=Proximal+Policy+Optimization+Algorithms
19. Meta-Learning with Memory-Augmented Neural Networks — Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap, 2016
https://scholar.google.com/scholar?q=Meta-Learning+with+Memory-Augmented+Neural+Networks
20. Learning to Learn by Gradient Descent by Gradient Descent — Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Nando de Freitas, 2016
https://scholar.google.com/scholar?q=Learning+to+Learn+by+Gradient+Descent+by+Gradient+Descent
21. Transformers learn in-context by gradient descent — Johannes von Oswald et al., 2022
https://scholar.google.com/scholar?q=Transformers+learn+in-context+by+gradient+descent
22. In-context Learning and Gradient Descent Revisited — Gilad Deutch et al., 2023
https://scholar.google.com/scholar?q=In-context+Learning+and+Gradient+Descent+Revisited
23. Low-Rank Few-Shot Adaptation of Vision-Language Models — Maxime Zanella and Ismail Ben Ayed, 2024
https://scholar.google.com/scholar?q=Low-Rank+Few-Shot+Adaptation+of+Vision-Language+Models
24. Meta-Adapter: An Online Few-shot Learner for Vision-Language Model — Cheng Cheng et al., 2023
https://scholar.google.com/scholar?q=Meta-Adapter%3A+An+Online+Few-shot+Learner+for+Vision-Language+Model
25. Cross-Domain Few-Shot Learning via Adaptive Transformer Networks — Naeem Paeedeh et al., 2024
https://scholar.google.com/scholar?q=Cross-Domain+Few-Shot+Learning+via+Adaptive+Transformer+Networks
26. Few-shot Adaptation of Multi-modal Foundation Models: A Survey — Fan Liu et al., 2024
https://scholar.google.com/scholar?q=Few-shot+Adaptation+of+Multi-modal+Foundation+Models%3A+A+Survey
27. AI Post Transformers: In-Context Learning as Implicit Learning Algorithms — Hal Turing & Dr. Ada Shannon, Wed,
https://podcast.do-not-panic.com/episodes/in-context-learning-as-implicit-learning-algorithms/
28. AI Post Transformers: NVIDIA: TTT-E2E: Unlocking Long-Context Learning via End-to-End Test-Time Training — Hal Turing & Dr. Ada Shannon, Sat,
https://podcast.do-not-panic.com/episodes/nvidia-ttt-e2e-unlocking-long-context-learning-via-end-to-end-test-time-training/
29. AI Post Transformers: Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts — Hal Turing & Dr. Ada Shannon, Tue,
https://podcast.do-not-panic.com/episodes/zero-shot-context-generalization-in-reinforcement-learning-from-few-training-con/
30. AI Post Transformers: A 2024 Survey Analyzing Generalization in Deep Reinforcement Learning — Hal Turing & Dr. Ada Shannon, Fri,
https://podcast.do-not-panic.com/episodes/a-2024-survey-analyzing-generalization-in-deep-reinforcement-learning/
Interactive Visualization: MAML and the Basics of Meta-Learning