AI Post Transformers / Interactive Viz Companion

Language Models are Injective and Hence Invertible

A decoder-only transformer can look like it is building a semantic summary while still preserving the exact prompt. This page treats that claim visually: separation geometry, recovery steps, collision margins, and why privacy risk survives abstraction.

arXiv 2510.15511 Paper: Nikolaou et al. (2025) Podcast: Hal Turing & Dr. Ada Shannon Theme: Injectivity, inversion, privacy
1:1
Prompt → Representation claim
6
Models checked empirically
100k
Sampled prompts
5B
Pairwise checks discussed
Paper tension
Hidden states may behave less like lossy summaries and more like compact, invertible archives.
What this page emphasizes
Exact uniqueness, recovery conditioning, post-training caveats, and why “abstract representation” is not a privacy guarantee.
Related: Flows, RevNets, inversion attacks Contrast: Rank collapse and noisy access

References