Neural Compression — The Codec That Learns

JPEG's quantization table and H.264's prediction modes were tables people worked out by hand. Neural compression hands that design job to the data instead. The two pillars — an autoencoder and an entropy model — the trick that gets around non-differentiable rounding, the hyperprior, and what standards bodies are actually arguing about on the road to JPEG AI.

End-to-end Optimized Image Compression

The metaphor: compiling a dictionary by hand, or learning one by reading

JPEG's quantization table is sixty-four numbers that engineers set one at a time, guided by psychophysical experiments on which spatial frequencies the human eye is bad at seeing. H.264's intra prediction modes and AV1's transform types came about the same way: somebody hypothesised that natural images contain a particular kind of structure, implemented it, and brought evidence to a standards meeting to win agreement. Thirty years of that.

Neural compression turns this upside down. Which transform to use, which components to throw away, and how to predict the next value are all learned from a large pile of images rather than decided by a person. Think of the difference between a dictionary compiled by hand and one learned by reading. The first can explain its own reasoning; the second cannot, but in exchange it soaks up the actual statistics of the data.

The three boxes every codec has

What's striking is that neural compression does not throw away the existing skeleton. JPEG, H.264 and a learned codec are all the same three boxes in a row.

1. Transform — stop working in pixels; move to a coordinate system where the information piles up in a few values
2. Quantization — round those values coarsely. This is where information is actually lost, and where "lossy" comes from
3. Entropy coding — replace the rounded values with bit strings, shorter for the values that occur more often

In JPEG the boxes hold the DCT, the quantization table and Huffman coding; in H.264, an integer transform, QP and CABAC. (The third box is the subject of Entropy Coding from Scratch.) What neural compression replaces is not the order of the boxes but their contents: the transform becomes a neural network, the probability prediction becomes a neural network, and only the rounding stays as it was. Everything below hangs off that picture.

What it really is: an autoencoder

The transform is handled by a network that squeezes an image into a small representation and expands it back — an autoencoder. Write the input image as $x$, the squeezing side (the analysis transform) as $g_a$, the expanding side (the synthesis transform) as $g_s$, and rounding as $Q$. The whole codec is this:

In words: turn the image into a stack of features with $g_a$, round those to integers, and turn them back into an image with $g_s$. The rounded integers $\hat{y} = Q(g_a(x))$ are literally what goes into the file.

Two things separate this from an ordinary autoencoder. The bottleneck is rounded to integers — you cannot write a real number in a finite number of bits — and shrinking the dimensionality is not the goal in itself. The goal is bits. If the skeleton looks familiar, Building a VAE from Scratch is where the resemblance gets explained.

Where the bits come from — the entropy model

So where is the bit count decided? An arithmetic coder or ANS writes the rounded values, but a coder that isn't handed a probability distribution cannot save a single bit. Supplying that distribution is the job of the second pillar: the entropy model $p_{\hat{y}}$.

Information theory says that writing down a value of probability $p$ costs about $-\log_2 p$ bits. So the bits needed for one image are

$$R = \mathbb{E}\left[-\log_2 p_{\hat{y}}(\hat{y})\right]$$

— the expected value of "how much probability the model managed to assign to the value that actually turned up." The better the prediction, the smaller the file. Improving compression ratio is improving prediction accuracy; there is nothing else to it.

What is being minimised — rate–distortion optimisation

If shrinking $R$ were the only goal, returning a black image would win. So it is added to a distortion term $D$ — usually MSE, or something more perceptual like MS-SSIM.

"Minimise bits and distortion combined, balanced by a weight $\lambda$." Raise $\lambda$ and the penalty on distortion grows, so the model spends bits to reconstruct faithfully. That $\lambda$ is what the user sees as the quality setting.

Here is the decisive difference from conventional codecs. JPEG and H.264 also do rate–distortion optimisation, but they do it per block at encode time, choosing among a fixed menu of modes. In a learned codec the same optimisation happens once, during training, over the entire codec — the transform and the entropy model are shaped together by a single loss. That is what "end-to-end optimised" means.