AI Post Transformers · Episode Companion

DualDecoder: Fixing GPU Memory Bloat in Long-Context KV Cache Offloading

Zuning Liang, Zhiyi Yao, Qi Chen, Yuedong Xu, Hao Dai, Zhiqiang Ding, Tongkai Yang, Jinlong Hou, Yuan Cheng — Fudan University, Shanghai Innovation Institute, Ant Group · July 2026

arXiv:2607.26475 → 64% GPU mem = bookkeeping, not KV up to 2.62× decode throughput 36.4% lower memory vs ShadowKV

References