AI Post Transformers — Episode Companion

Naive Test-Time Adaptation Destabilizes LLM Predictions

📄 arXiv:2602.09719 Longhuan Xu, Cunjian Chen, Feng Yin — 2026 SCALENET · Hypernetwork LR Control Llama-3.3-70B · Qwen3-32B

SCALENET is a hypernetwork fix for unsupervised test-time adaptation in LLMs. A 70B Llama model's negative log-likelihood balloons from 2.21 to 11.49 after five naive adaptation steps — this page visualizes why single-sample gradients blow up, how a per-layer per-step learned control signal stabilizes them, and where the fix still fails on open-ended reasoning tasks.

Unsupervised, sample-specific TTA: the adapt-and-reset loop

Each prompt gets its own gradient episode, on the prompt's own NLL — no gold label, no memory carried to the next prompt.

Frozen backbone stays untouched. Only LoRA Q/V matrices in attention layers absorb the K-step update, then it is discarded before the next prompt arrives.

The gradient the update actually has to serve

You only ever compute ∇NLL(x). The bet is that it correlates with ∇NLL(y | x). SCALENET's whole job is keeping that inner product non-negative — without ever seeing y.

Llama-3.3-70B negative log-likelihood across 5 TTA steps

A single global learning rate cannot serve both regimes: too small and nothing moves, too large and it explodes.

Fixed-rate baseline Step-wise learned LR Layer-wise learned LR (SCALENET)

Why one prompt can't average out its own noise

Batch training cancels idiosyncratic gradient directions across thousands of examples. A single-sample TTA step has no such averaging — hover the bars.

SCALENET: a per-layer, per-step learning-rate controller

A 2-layer MLP (hidden=128) reads a compressed prompt representation and outputs a non-negative multiplier for every layer × step × {Q, V} pair.

Training vs. deployment

Supervision moves upstream — it doesn't disappear.

TTA lineage

Where SCALENET sits relative to prior unsupervised TTA methods.

NLL at step 5 — Llama-3.3-70B

Layer-wise control isn't a marginal fix — it's the difference between collapse and stability.

ROUGE across datasets — where it holds, where it breaks

Extractive tasks (prompt already contains the answer) climb steadily. Open-ended reasoning tasks regress below the no-TTA baseline (dashed line). Points below the baseline are flagged in orange/red.

Fixed-rate Step-wise Layer-wise (SCALENET) No-TTA baseline (dashed)

Learned per-layer, per-step multiplier magnitude

No clean monotonic story — neighboring layers can differ by orders of magnitude. The one consistent pattern: magnitude peaks at step 1, then decays fast.

Hover any cell for its exact learned multiplier. Rows are representative attention layers; columns are adaptation steps 1–5.

References

Unsupervised Layer-Wise Dynamic Test Time Adaptation for LLMs — Longhuan Xu, Cunjian Chen, Feng Yin (2026)

arxiv.org/abs/2602.09719

Test-Time Training with Self-Supervision for Generalization under Distribution Shifts — Sun, Wang, Liu, Miller, Efros, Hardt (2020)

Google Scholar

Tent: Fully Test-Time Adaptation by Entropy Minimization — Wang, Shelhamer, Liu, Olshausen, Darrell (2021)

Google Scholar

Test-Time Training on Nearest Neighbors for Large Language Models — Hardt, Sun (2024)

Google Scholar

The Surprising Effectiveness of Test-Time Training for Abstract Reasoning — Akyürek, Damani, Qiu, Guo, Kim, Andreas (2024)

Google Scholar

Test-time Learning for Large Language Models — Hu, Zhang, Chen, Wen, Shuai, Luo, Xiao, Li, Tan (2025)

Google Scholar

SLOT: Sample-specific Language Model Optimization at Test-time — Hu, Zhang, Fang, Chen, Wang, Zhang, Qi (2025)

Google Scholar

COME: Test-time Adaption by Conservatively Minimizing Entropy — Zhang, Bian, Kong, Zhao, Zhang (2024)

Google Scholar

Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models — Rannen-Triki, Bornschein, Pascanu, Hutter, et al. (2024)

Google Scholar

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML) — Finn, Abbeel, Levine (2017)

Google Scholar