Hydragen: Shared-Prefix Attention for High-Throughput LLM Inference

Juravsky, Brown, Ehrlich, Fu, Ré, Mirhoseini · 2024 · exact attention via prefix/suffix decomposition