Optimizing Verification and Efficiency in Multi-Draft Speculative Decoding

These sources explore advanced techniques for accelerating Large Language Model (LLM) inference through speculative decoding, a process where smaller "draft" models predict tokens for a larger "target" model to verify in parallel. A primary focus is Multi-Draft Speculative Decoding (MDSD), which uses multiple draft sequences to increase the probability of acceptance and reduce latency. Researchers have introduced SpecHub to simplify complex optimization problems into manageable linear programming, while others utilize optimal transport theory and q-convexity to reach theoretical efficiency upper bounds. Additionally, the Hierarchical Speculative Decoding (HSD) framework stacks multiple models into a tiered structure, allowing each level to verify the one below it. Collectively, these papers provide mathematical proofs, sampling algorithms, and hierarchical strategies designed to maximize token acceptance rates and minimize computational overhead.