Where does the task live, and who pays for it? Click a lane.
Counts attended example tokens only, not latency or memory. Example length fixed at 1024 (the hypernetwork's max example input); the Gist prefix is 16 tokens.
Rows = query token, columns = key token it may attend to. Hover a cell.Illustrative: 6 example, 4 Gist, 3 input tokens (paper uses 16 Gists)
Read out the Gist tokens' key and value states at every layer and prepend them to a frozen copy of the model.
Single Llama-2-7B, 4-bit quantized (QLoRA). Flip the LoRA adapter.
Gists computed near the end of the examples (e.g. position ~800) need a matching shift downstream.
Step through the recipe.
Numbers as reported in the episode from the paper. No seeds or confidence intervals are reported.
Hover cells for detail.
Only one Gist length (16) and one model size were evaluated. Everything else is unknown.