KV Cache Translation Across Heterogeneous Large Language Models

Jin-woo Lee, Minkyung Song, Junghyun Oh, Seunghoon Han, Gwangseon Jang, Soyoung Park, Sungsu Lim · Chungnam National University, KISTI, UNIST · 2026
arXiv 2607.28979 Interactive Visualization

Five-step inference pipeline

Click a step or press Next. The target never reads the original text at inference.

Why caches don't just transfer

A Llama cache can't be loaded into Gemma or Qwen. Illustrative schematic — no real dimensions

Two competing errors → a U-shaped loss

Slide the translation window from early to late layers. Illustrative curve shape; paper's U is on gpt2→gpt2 only

Where the improvement comes from

Last-state shift = correction deficit + orthogonal component. Section 4.3: mostly translation and orthogonal shift shrink. Qualitative heights, not measured

Mixture-of-Translators: token-level routing

Click a token. Gate scores pick the top-K translators; weights are renormalized. Illustrative gate scores
Top-K:

Ablation: Qwen2.5 7B → 0.5B

Reported No variance reported; gaps are 2–3 points.

Results versus native prefill

Reported Native = target model prefilling the context itself. Higher is better.

Recovery ratio per cell

Derived: translated ÷ native from reported cells A faithful translator should at best tie 100%.

Three controls that would settle "beats native"

Click a control to see what each outcome would mean.

Scope of the evidence

Covered vs. not tested, per the hosts.

Peak memory with several translators

Reported Small Qwen model, GiB. Flat multi-agent memory exists only in Free mode, which evicts non-hub caches.

Setup at a glance

Reported Hover cells.

References

  1. KV Cache Translation across Heterogeneous LLMs — Lee et al., 2026 (arXiv 2607.28979)
  2. Cache-to-Cache — Fu et al., 2025
  3. DroidSpeak — Liu et al., 2024
  4. Communicating Activations Between Language Model Agents — Ramesh, Li, 2025
  5. CacheBlend — Yao et al., 2024
  6. PagedAttention — Kwon et al., 2023
  7. Distilling the Knowledge in a Neural Network — Hinton, Vinyals, Dean, 2015
  8. FitNets — Romero et al., 2015
  9. On-Policy Distillation of Language Models — Agarwal et al., 2024
  10. Cartridges — Eyuboglu et al., 2025
  11. Latent space communication via K-V cache alignment — Dery et al., 2026
  12. KVComm — Shi et al., 2026
  13. Interlat — Du et al., 2026
  14. Model stitching — Bansal, Nakkiran, Barak, 2021
  15. DAgger — Ross, Gordon, Bagnell, 2011