AI POST TRANSFORMERS / VISUAL COMPANION ARXIV 2605.10938 LATE DISCRETIZATION

ELF and Continuous Language Diffusion

A visual walkthrough of the claim that continuous-time text generation gets better when the model stays in embedding space almost all the way through sampling. The page focuses on mechanism, comparative dynamics, and where the causal story is still confounded.

Core Claim
Do not snap back to tokens at every denoising step.
Reported Result
105M ELF beats 170M baselines on OpenWebText perplexity.
Why It Matters
Imports flow matching and guidance tricks from image diffusion.
Main Skepticism
Were gains caused by the objective, or by bundled recipe changes?

At a Glance

CONTINUOUS TRAJECTORY VS TOKEN SNAPBACK
steps: 8-128

Overview Pipeline

The left lane shows token-bound diffusion repeatedly decoding through the vocabulary. The right lane shows ELF staying continuous until a single end-of-trajectory commit.

TAB 1 / SYSTEM VIEW

Sampling Timeline

One path keeps re-discretizing. The other keeps its state soft and only decodes once.

WHO COMMITS WHEN?

Commit Pressure

Mock entropy traces illustrate how early token commitments can force coarse decisions before the latent path has settled.

LOWER IS MORE COMMITTED

Latent Dynamics Lab

Step through a synthetic embedding trajectory. Hover cells to inspect where semantic mass is still diffuse versus where the model has already started collapsing into token-like corners.

TAB 2 / INTERACTIVE HEATMAP
diffuse latent mass sharpening features near discrete commitment
Synthetic dimensions and confidence values illustrate the paper’s design intuition rather than reproducing raw model internals.

Results Explorer

Switch between generative quality, step budget, and transfer-task behavior. Values are visualized from the episode’s reported relationships, with mock interpolation where the transcript is qualitative.

TAB 3 / COMPARISON CHARTS
105M
ELF parameter count highlighted in the episode.
170M
Baseline scale used for the headline contrast.
10×
Fewer training tokens in the reported OpenWebText setup.
1×
Single final discretization instead of repeated snapback.
ELF 105M discrete diffusion 170M continuous baselines

Skeptic Lens

The paper argues for late discretization, but the episode presses on confounds. This section separates the visible evidence from the invisible recipe bundle.

TAB 4 / CLAIM DECOMPOSITION

References

Key papers discussed or implied by the episode, with direct arXiv links where available and source links for the rest.

EXTRACTED ARXIV IDS: 2605.10938