autoregressive

Definition

Generating a sequence one element at a time, each step conditioned on everything produced so far — the factorisation of a joint distribution into a product of conditionals. It is how GPT-style language models both train and sample. The cost is serial generation: you cannot produce token n before token n−1, which is why diffusion and parallel-decoding schemes keep being explored.

Explainers using this term

• Paper Explained: JoyAI-Video-Edit — Autoregressive Diffusion That Keeps Editing 720p Video at ~30 FPS, Without Ever Seeing the FutureJoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion