continuous-latent

Definition

Representations kept as real-valued vectors rather than quantised into a discrete codebook. Continuous latents lose nothing to quantisation and keep gradients flowing, but they do not slot into the machinery built around next-token prediction over a vocabulary, which is why discrete tokenisers such as VQ-VAE remain common despite the information they discard. The choice between the two is a recurring design fork in audio, image, and video generation.

Explainers using this term

• Paper Walkthrough: AURORA-LM — A Diffusion Language Model That Keeps Its Text Latent Full-Width Instead of Compressing ItAURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling