Diffusion Models from the Ground Up — Add Noise, Then Subtract It

A reading of the DDPM paper (Ho et al., 2020) grounded strictly in its own text: the forward process that adds noise and the reverse process that removes it, why the objective settles into 'predict the noise', how a simplified loss hurt likelihood while improving samples, and what the paper does and does not say about step count.

Denoising Diffusion Probabilistic Models

Retracing the path that destroyed the picture

A diffusion model is a parameterized Markov chain trained using variational inference to produce samples matching the data after finite time (§1). Its transitions are learned to reverse a diffusion process — a Markov chain that gradually adds noise to the data until the signal is destroyed.

The design hinge sits right there. As the paper puts it, when the diffusion consists of small amounts of Gaussian noise, it is sufficient to set the sampling chain transitions to conditional Gaussians too, which allows a particularly simple neural network parameterization (§1). Break the destruction into fine enough steps and the reconstruction becomes writable in a simple form.

The forward process: shrink, then add

The forward process is not learned. It is fixed to a variance schedule $\beta_1,\dots,\beta_T$ (Eq. 2):

Swapping the symbols for plain words: $q(\mathbf{x}_{t}|\mathbf{x}_{t-1})$ is the rule that makes the next, slightly dirtier picture out of the one you are holding, and $\mathcal{N}$ is the instruction which says draw from a bell curve — the two slots after the semicolon being where that bell curve sits and how wide it is. In words: at each step, scale the image down by $\sqrt{1-\beta_{t}}$ first, then add noise of variance $\beta_{t}$. Fade the picture a little, sprinkle grit over it — that is the whole step. The scaling is there so that variance does not grow as noise accumulates (Appendix C).

The process has a convenient property. Writing $\alpha_{t}=1-\beta_{t}$ and $\bar{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$, the state at any timestep has a closed form (Eq. 4):

Here $\bar{\alpha}_{t}$ is a single number standing for how much of the original picture is still present after $t$ steps, and $1-\bar{\alpha}_{t}$ is how much of it has been replaced by noise. Read in words, the formula says the bell curve is centred on the original faded to $\sqrt{\bar{\alpha}_{t}}$ of its strength, and its width is exactly the part that has been given over to noise. That is, you can jump from the original image to any step in one shot — no thousand-iteration loop — so training can sample $t$ at random (§2). The experiments use $T=1000$ with $\beta$ increasing linearly from $10^{-4}$ to $0.02$. Those constants were chosen to be small relative to data scaled to $[-1,1]$, keeping the forward and reverse processes approximately the same functional form while pushing the signal-to-noise ratio at $\mathbf{x}_{T}$ as low as possible (§4).

The reverse step looks like a small step downhill

The reverse process is a chain of learned Gaussian transitions starting at $p(\mathbf{x}_{T})=\mathcal{N}(\mathbf{0},\mathbf{I})$ (Eq. 1), trained on the variational bound on negative log likelihood (Eq. 3).

A correspondence the paper points out matters here. Under the $\boldsymbol{\epsilon}$-prediction parameterization introduced below, the sampling procedure (Algorithm 2) resembles Langevin dynamics, with $\boldsymbol{\epsilon}_{\theta}$ acting as a learned gradient of the data density (§3.2). One step is close to "move a little along a gradient, then add noise".

Unfolding the variational bound

The bound is rewritten for variance reduction (Eq. 5) into three kinds of terms: $L_{T}$, the $L_{t-1}$ terms, and $L_{0}$. Because the $\beta_{t}$ are fixed constants, $L_{T}$ is a constant during training and can be ignored (§3.1). The middle terms become tractable once the forward process posterior is conditioned on $\mathbf{x}_{0}$ (Eqs. 6 and 7), so every KL divergence is a comparison between Gaussians and has a closed form (§2).

From there $L_{t-1}$ reduces to a squared error between $\tilde{\boldsymbol{\mu}}_{t}$ and $\boldsymbol{\mu}_{\theta}$ (Eq. 8). Read plainly, the network should just predict the posterior mean. That would be the natural stopping point — but the paper rewrites it once more.