Latent Space Communication via K-V Cache Alignment

This episode explores a Google DeepMind paper proposing that separately trained language models can exchange raw internal state — the key-value cache built during transformer inference — through a shared "global latent space," rather than communicating only through text. The hosts unpack why text is a lossy bottleneck for inter-model communication, and how lightweight, frozen-weight adapter pairs let each model translate its own cache into and out of this common space, keeping training cost linear rather than combinatorial as more models join the pool. A striking result anchors the discussion: translating a model's cache through this shared space can sometimes outperform the model's own untouched cache on the same task. The conversation connects this idea to familiar concepts like prefix-tuning and continuous latent reasoning, framing the cache exchange as a dynamic, evolving version of a static soft prompt. Listeners interested in how models might one day share "trains of thought" instead of finished sentences will find the tension between the approach's architectural simplicity and its surprising performance gains especially compelling.