Why JPEG Degrades — The DCT and Quantization from Scratch

When a photo shrinks to a tenth of its size, what exactly was thrown away? Colour conversion, 8×8 blocks, the DCT, the quantization table and the zigzag scan — followed by what the number in "quality 90" really is, and where blocking and mosquito noise are born. Ending with the decisions people who ship images actually make.

The metaphor: sort before you discard

Nobody halves their belongings by throwing out boxes at random. You sort first — daily use, occasional use, probably never again — and then discard from the back. The amount discarded is the same; the pain is an order of magnitude smaller.

JPEG does precisely this. It reorders an image from "components the eye is sharp about" to "components the eye is blunt about", then discards from the back. The tool that reorders is the DCT; the act of discarding is quantization.

The whole pipeline: only five stages

1. Colour conversion — regroup RGB into one brightness channel and two colour channels (YCbCr)
2. Chroma subsampling — lower the resolution of the two colour channels only
3. 8×8 blocks and the DCT — rewrite each block as a list of "wave strengths"
4. Quantization — round those strengths onto a coarse scale
5. Zigzag scan and entropy coding — pack the rounded result losslessly

Only stages 2 and 4 are lossy. Stages 1, 3 and 5 are in principle reversible. Everything people call "JPEG degradation" comes out of those two places.

Separating brightness from colour

Human vision is sharp about changes in brightness and blunt about changes in colour. To exploit that asymmetry, JPEG first regroups RGB into $Y$ (luma) and $C_b$, $C_r$ (blue-ish and red-ish chroma).

The same thing in words: take a pixel's red, green and blue values, weight each one by how bright that colour looks to human eyes, and add the three together. The $Y$ that comes out is a single number meaning "how bright this pixel is".

What this formula says, plainly, is that green looks brightest and blue looks darkest to us, and those sensitivities are used directly as the weights. The chroma channels are $C_b \propto B - Y$ and $C_r \propto R - Y$ — "the colour left over once brightness is removed". After the move to YCbCr the information concentrates in $Y$, while $C_b$ and $C_r$ cluster near zero.

Chroma subsampling: half the colour resolution

Next JPEG throws away chroma samples only. The notation is 4:2:0 (halved in both directions, so a quarter of the colour samples), 4:2:2 (halved horizontally) and 4:4:4 (untouched).

At 4:2:0 the data per pixel is 1 for $Y$ plus 0.5 for $C_b$ and $C_r$ combined, so 1.5 bytes against RGB's 3 — half, before the DCT or quantization has been touched at all. On landscape photography you will not notice. On thin red lines or coloured text you will: the colour outlines smear.

8×8 blocks and the DCT

Each component is cut into 8×8 pixel blocks, and from here on everything is done per block, with no reference whatsoever to neighbours. That independence is what later produces blocking artefacts.

Each block goes through the discrete cosine transform (DCT). The name is intimidating; the job is not.

The same idea in words: pick one of the 64 stripe patterns, hold it against the block, and ask "how much does this block look like this stripe?" — then write the answer down. $u$ and $v$ say which stripe you picked (how many stripes across, how many down), and the double sum is the asking, carried out over all 64 pixels of the block. Repeat for all 64 stripes and the transform is finished.

$f(x,y)$ is the original pixel value, $F(u,v)$ the transformed coefficient, and $C(0)=1/\sqrt{2}$ with all other $C$ equal to 1. Do not be put off by the size of it: it rewrites 64 pixel values as the strengths of 64 stripe patterns, nothing more.

Those 64 patterns (the basis) start at top left with a flat fill; going right the horizontal stripes get finer, going down the vertical stripes get finer. The top-left $F(0,0)$ is the DC coefficient, which is simply the block's average brightness. The remaining 63 are AC coefficients, describing finer variation.

Here is the crucial point. The DCT discards nothing. It rewrites 64 numbers as 64 other numbers, and the inverse transform recovers the original exactly — an orthogonal transform, a change of basis in the sense of the linear algebra article. So why bother? Because in natural images the energy concentrates in a handful of top-left coefficients. In a photo with large smooth regions the bottom-right high-frequency coefficients are already close to zero. The reordering gathers everything discardable into one corner — that is the DCT's single, entire job.