Students Professional
JA EN

#speculative-decoding

1 articles

01 ·Inference & Serving·★ MEMBER·PAPER·8 min read Speculative Decoding from Scratch — How a Tiny Draft Model Speeds Up an LLM Without Changing a Single Output Speculative decoding lets a small model draft several tokens ahead while a large model verifies them in one batch — provably identical outputs, 2–3x faster in practice. A from-scratch walkthrough of the mechanism and the intuition behind the acceptance rate α, straight from the original paper.