Instead of paying the long-context bill on every question, this page shows the paper’s core wager:
compile a document once into a compact LoRA adapter, then answer later with the source text gone.
Prompting keeps re-reading. Context distillation updates per document. Doc-to-LoRA learns a forward map that emits
the update directly.
Hover cells and nodes. The visuals contrast where the document lives:
tokens, transient cache, or a small generated adapter.
Perceiver-Style Hypernetwork
The document is chunked, cross-attended into a fixed latent array, and decoded into low-rank matrices.
Step through the transformation below.
Rank and Chunking Interaction
Chunking lets the system accumulate richer adapter structure without changing the hypernetwork’s external output interface.
Mock matrices visualize activation coverage across the generated low-rank update;
warmer cells indicate stronger document-specific adaptation.
Repeated-Query Economics
The paper’s systems argument is not “prompts are obsolete.” It is that repeated follow-ups can shift the cost winner
once the document has already been compiled.
Accuracy Against Length
Synthetic long-context retrieval is where the headline looks sharp: performance stays high past 32K tokens because the
downstream model no longer needs to scan the full source at answer time.
Doc-to-LoRA Prompting Context Distillation
Break-Even Query Heatmap
The real decision variable is not a benchmark row. It is where document length and downstream query count cross the
threshold that justifies compiling context into weights.
Storage Formats in Tension
Doc-to-LoRA competes with compressed prompts, persistent caches, retrieval systems, and other one-shot adapter schemes.
This map emphasizes what each format optimizes.
The caution from the episode is preserved here: limited-query evaluation does not settle whether weights are the best
long-term home for reusable memory.
References
Compact pointers for the design space around instant adaptation, hypernetworks, prompt compression, and persistent memory.