Gemma 3 on Ukrainian: 2.98 → 4.44 chars/token with a Ukrainian-focused vocabulary. The authors call it almost a 50% inference speedup; latency and memory were not measured.
Why the model is welded to its vocabulary
Token IDs index the input embeddings and the output head. The body learned to read those exact vectors. Illustrative cell colors
Two ways to fill the new rows
Tap or hover a node.
Step by step: aligning two tokenizations
What gets matched Illustrative values
Attention output = Σ over earlier tokens of (attention weight × value vector), taken before the output projection. Paper does not say which variant gave the headline results.
One training step
Reported results
Layer ablation (Gemma 4B, Belebele, single runs)
Values read from the discussion as approximate (~). All-layers also costs about 2.5× the memory. Nobody tested all-layers at depth 12.
Where the loss sits
Three candidate explanations for the all-layers failure