Four residual streams; each layer's mixer is forced to be doubly stochastic.
exp(W), then alternate column and row normalization. mHC stops at 20 sweeps, ending on a row step.
Corners are permutations; the uniform matrix sits in the middle.
Entry ranges of single mixers vs. the composite over 30 to 90 layers, and the backward-gain drift.
The average stream passes with gain exactly 1. The difference channel shrinks by σ₂ per layer, σ₂ ≤ 1 − n·(min entry).
Each layer = λP + (1−λ)U with a random permutation P (n=4), the case where the bound is exact (λ = 1 − 4m). Composite = λᴸ·ΠP + (1−λᴸ)·U.
Four stream values before and after L layers. The mean never moves; spread decays.
Mean channel gain = 1 for any doubly stochastic mixer. Difference gain ≤ (1 − n·m)ᴸ. Dashed line: horizon 1/(1−σ₂).
Backprop through Sinkhorn gives the Fisher-Rao gradient. Training updates the logits, so the metric enters again.
q = off-diagonal entry. Fisher-Rao: q′ = −q(1−q). Logit flow: q′ ∝ −q². Start q₀ = 0.3.
Slide the target distance ε to the corner.
Four streams on a laptop CPU, random logits or hand-built linear loss, exact projections. Not a trained model.
Sweeps needed for column sums accurate to 1e-3, vs. mHC's 20-sweep budget.
Near the answer each sweep shrinks error by σ₂², the same σ₂ that sets the horizon. Slide the memory horizon h = 1/(1−σ₂).
Hover a cell. Every check is a simulation of a proved theorem; no trained mHC was measured.
Birkhoff mixers vs. isometric mixers that fix the ones vector (group ≅ O(n−1)).