Values as stated in the episode: ~20 Chinchilla, ~1,000 the paper's runs, ~2,000 Llama-3-8B, ~100,000 extrapolated.
PTQ damage vs data Illustrative
Paper: loss increase from rounding is a power of D/N, exponential as bits drop, smaller for bigger N. Shapes only; constants here are invented for display.
Penalty heatmap: bits × D/N Illustrative
Hover a cell.
Hatched columns lie beyond the ~1,000 D/N the paper's runs reach: power-law extrapolation only.
Paper's weight-only form. γ below is a slider, not a fitted value.
Bits vs parameters trade Illustrative
Real parameter count needed to hold N_eff = 1 (relative units) at each bit width, same γ as above.
Fixed compute, lower precision: C ≈ 6·N·D·P/16
Exact cost relation from the paper; axes are relative (log2), no measured values.
Cost model vs hardware Illustrative
Episode: H100 FP8 ≈ 2× BF16; FP6 has no native tensor-core path; INT7 is not a speed class. Step view is a sketch of that argument, untested in the paper.
Model size needed to hold loss (as stated in episode)