AI Post Transformers • Interactive Visualization Companion

Learning in Random Nets and Generalization

Minsky’s 1961 argument is visualized here as a clash between two stories: reward can strengthen a path, but only a good representation can make nearby situations count as meaningfully similar. The page stays diagram-first: random topology, similarity geometry, transfer-vs-interference, and the later lineage from perceptrons to random features and echo state networks.

1. Reward Is Not a Metric

The overview diagram separates four learning stories from the transcript: rote memory, associative recall, pattern recognition, and true generalization. Hover any block or connection to see what the random-net proposal can plausibly do and where it fails.

plausible with local reinforcement needs a useful similarity structure modern disciplined descendant

2. Similarity Geometry

The heatmap and scatter view show the paper’s central objection. Reward can mark a successful response after the fact, but if the hidden geometry does not cluster related cases, spillover becomes interference rather than generalization.

local neighborhood coherence ambiguous overlap high interference zone

3. Random Scaffold Lab

This lab compares three mock regimes: pure internal reinforcement in a random tangle, fixed random features with a trained readout, and a learned representation. Toggle to see which setup converts scale into useful separation instead of expensive ambiguity.

task accuracy transfer quality interference cost

4. Historical Lineage

The timeline maps what survived. Randomness persisted, but only after later work isolated the random part from the trainable part, or wrapped it inside a precise objective, memory system, or optimization rule.

representation learned explicitly random scaffold retained memory / retrieval externalized

References

M. Minsky et al., 1961 • primary paper
Rosenblatt, 1958 • cleaner classifier story
Samuel, 1959 • explicit evaluation and selective improvement
Jaeger, 2001 • random recurrent reservoir with trained readout
Rahimi & Recht, 2007 • fixed random map, explicit learner
Cover & Hart, 1967 • similarity only works when the space is right
Cortes & Vapnik, 1995 • geometry plus margin, not vague reward
Wang et al., 2023 • retrieval as a feature, not a failure