Subliminal Learning: Why Noise Carries Hidden Knowledge

This episode examines "Learning Through Noise: Why Subliminal Learning Works and When It Fails," which investigates how a student network can absorb a teacher's knowledge purely by imitating its outputs on random, task-irrelevant noise—no labeled data required. The discussion traces this back to 2025 research showing hidden traits like animal preferences can transfer between models through seemingly innocuous number sequences, raising alarms for the now-ubiquitous practice of knowledge distillation. It unpacks the prevailing explanation—a proven theorem that shared weight initialization between teacher and student drives this transfer—and previews how the featured paper challenges that assumption using a cleverly designed MNIST experiment with surgically split output heads to isolate the mechanism. The stakes are significant: if the "dark knowledge" channel isn't confined to near-identical model pairs, biases, misalignment, or backdoors could silently propagate through distilled models in ways current data-filtering practices can't catch. Listeners interested in AI safety, model distillation pipelines, or the mechanics of how neural networks transmit information beyond their explicit training signal will find the paper's methodology and its safety implications compelling.

Sources:
1. Learning Through Noise: Why Subliminal Learning Works and When It Fails — Vincent C. Brockers, Roman D. Ventzke, Valentin Neuhaus, Belén Hidalgo-Ogalde, Viola Priesemann, 2026
http://arxiv.org/abs/2605.23645
2. Distilling the Knowledge in a Neural Network — Geoffrey Hinton, Oriol Vinyals, Jeff Dean, 2015
https://scholar.google.com/scholar?q=Distilling+the+Knowledge+in+a+Neural+Network
3. Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Data — Alex Cloud, et al., 2025
https://scholar.google.com/scholar?q=Subliminal+Learning%3A+Language+Models+Transmit+Behavioral+Traits+via+Hidden+Signals+in+Data
4. Knowledge Distillation: A Survey — Jianping Gou, Baosheng Yu, Stephen Maybank, Dacheng Tao, 2021
https://scholar.google.com/scholar?q=Knowledge+Distillation%3A+A+Survey
5. Planting Undetectable Backdoors in Machine Learning Models — Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, Or Zamir, 2022
https://scholar.google.com/scholar?q=Planting+Undetectable+Backdoors+in+Machine+Learning+Models
6. Language models transmit behavioural traits through hidden signals in data — Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Sören Mindermann, Jacob Hilton, Samuel Marks, Owain Evans, 2026 (Nature)
https://scholar.google.com/scholar?q=Language+models+transmit+behavioural+traits+through+hidden+signals+in+data
7. Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs — Jan Betley, Daniel Chee Hian Tan, Niels Warncke, Anna Sztyber-Betley, Xuchan Bao, Martín Soto, Nathan Labenz, Owain Evans, 2025 (ICML)
https://scholar.google.com/scholar?q=Emergent+misalignment%3A+Narrow+finetuning+can+produce+broadly+misaligned+LLMs
8. Sleeper agents: Training deceptive LLMs that persist through safety training — Evan Hubinger et al., 2024
https://scholar.google.com/scholar?q=Sleeper+agents%3A+Training+deceptive+LLMs+that+persist+through+safety+training
9. Shared global and local geometry of language model embeddings — Andrew Lee, Melanie Weber, Fernanda Viégas, Martin Wattenberg, 2025
https://scholar.google.com/scholar?q=Shared+global+and+local+geometry+of+language+model+embeddings
10. Training-free tokenizer transplantation via orthogonal matching pursuit — Charles Goddard, Fernando Fernandes Neto, 2025
https://scholar.google.com/scholar?q=Training-free+tokenizer+transplantation+via+orthogonal+matching+pursuit
11. Prevalence of neural collapse during the terminal phase of deep learning training — Vardan Papyan, X. Y. Han, David L. Donoho, 2020 (PNAS)
https://scholar.google.com/scholar?q=Prevalence+of+neural+collapse+during+the+terminal+phase+of+deep+learning+training