The pipeline: compact before the questions exist
Click any box. Weights stay frozen; only KV slots are trained for one document.
Click a box in the pipeline.
Why the cache is the bottleneck
Slots per layer per KV head grow linearly with context. Arithmetic only: N0 and N0/r. Counts pairs, not bytes.
context N0:
Select vs. create
Eviction keeps real key-value pairs. Latent compaction can place a slot where no token's key sits.Illustrative
Query-aware vs. query-agnostic
The paper compresses before any question arrives. Toggle the setting.
Building the cache, step by step
Example from the episode: 4K document at 32x gives B = 128 slots per layer per KV head.
Why match the softmax mass?
Question tokens join the document under one softmax. Drag the document's share.Illustrative
doc mass share:
Reported results
Only numbers stated in the episode. Single model, one seed per document, no confidence intervals.
Compression-ratio axis (what was tested)
Phonebook. Positions are ordinal, not scaled to measured accuracy.
Evidence map: what the paper does and doesn't show
Hover or tap a cell for detail.
Hover a row.
Which RULER tasks carry the average?
Categories from the episode. Per-task values exist only as plots in Appendix E.Illustrative