CFG and Samplers — What the "Strength" Knob in Generative AI Really Does

What "CFG Scale" and "Sampling steps / method" actually do inside an image generator, explained from zero. CFG extrapolates along the difference between the conditional and unconditional prediction; the sampler is a numerical ODE solver. Both knobs trade something away.

Classifier-Free Diffusion Guidance

The two knobs every generator has

Open any image-generation UI and you will find two kinds of knob under the prompt box: CFG Scale (guidance scale), and Sampling steps / Sampling method. Turn the first one up and the picture follows the prompt more closely — until the colors burn out. Turn the second one down and generation gets faster — until the picture falls apart. Most people settle on "about 7" and "about 25 steps" by feel and never learn why.

The two do completely different jobs. CFG is an extrapolation: it pushes the prediction further along the direction the condition is pulling. The sampler decides how many steps you take along a fixed trajectory. The first trades faithfulness against diversity; the second trades quality against time. And they are not independent — move one and the right value for the other shifts too.

Background: generation is repeated noise-guessing

The internals of diffusion are covered in Diffusion Models from Scratch. Only one fact is needed here. Hand the network a noisy image and it predicts "this is the noise mixed into it." Subtract a little of that prediction, repeat, and static turns into a picture. Call that prediction $\varepsilon_\theta$.

For conditional generation a prompt $c$ joins the input. Now $\varepsilon_\theta(x_t, c)$ means "the noise in this static, given that we're making a picture of a cat." In principle that alone is enough, and it does work — but it is not faithful enough to the prompt. Ask for "a red cube on a blue table" and you tend to get something vaguely reddish, because the model is pulled toward the average of its training data.

The intuition: trust only the difference between two answers

So classifier-free guidance (CFG) asks the same network twice: once with the prompt shown, giving $\varepsilon_{\text{cond}}$, and once with the prompt hidden — an empty string — giving $\varepsilon_{\text{uncond}}$.

Their difference $\varepsilon_{\text{cond}} - \varepsilon_{\text{uncond}}$ isolates exactly what changed because the model saw the prompt. Everything the model would have produced anyway cancels in the subtraction; only the direction the condition pushes survives. So why stop at pushing one unit along that direction?

In words: start from the unconditional answer and walk $s$ times as far toward the conditional one. At $s=0$ the condition is ignored; at $s=1$ the algebra collapses to plain $\varepsilon_{\text{cond}}$, i.e. ordinary conditional generation; at $s>1$ you overshoot the conditional answer and keep going. "CFG Scale 7.5" in the UI simply means $s = 7.5$.

Why overshooting increases faithfulness

Equation (1) is, underneath, a sharpening operation. The predicted noise is tied (up to sign) to the gradient of the log density — the score — and rewriting the CFG combination through that correspondence shows it is equivalent to sampling from:

In words: take the plain conditional distribution and multiply it by "how much this image looks like the prompt," raised to the power $s-1$. With an exponent above 1, prompt-like regions get lifted and everything else gets pushed down — exactly what lowering the temperature did in the figure above. The peak sharpens and the tails thin out.

Which means the gain in faithfulness and the loss of diversity are not cause and side effect — they are the same operation seen from two sides. "More faithful and more varied" is not something a single $s$ can give you.