LayerRoPE: Treating Transformer Depth Growth as a Positional Code
Shikhar Srivastava & Christopher Kanan · Univ. of Rochester · 2026 preprint (under review)
arXiv:2610.09179
16
open LLMs studied
99% / 1%
shared / outlier dims
143°
MDS arc, LLaMA-2 7B
3.4×
less compute vs Pre-Norm
≤0.013%
FLOP overhead
~10×
LR tolerance range
Paper numbers are labeled with their source. Concept diagrams and toy curves are illustrative, not measurements from trained models.
References