Live context length as an agent works. Schematic shapes only; no values are paper data.
Why the paper splits the root question Illustrative
Why later tokens cost more Illustrative
Each new token reads the KV entries of every earlier token. Hover a row to count keys read.
Hover a cell.
Measured H100 throughput gap, 16K vs 256K Measured
Peak decode throughput before KV eviction. Repeat runs varied ≤2%. Qwen's "~7" is approximate in the episode.
From measurement to "estimated optimized retail cost" Modeled
Straight line from full-window retail price down to 0.20× at 16K, clamped below 16K (Appendix B as described in the episode). Neither closed model was measured by depth.
Same tokens, two price lists
Accuracy change ranges Benchmark
Bars span the reported min to max change in points vs full-context baseline. Single runs, no confidence intervals.