From H.264 to AV1 — What a Codec Generation Change Really Involves

H.264 has ruled video for two decades; AV1 arrived promising royalty-free efficiency. We unpack the transition along three axes — coding efficiency, compute cost, and licensing — and end with a practical framework for deciding which codec to ship, starting from zero background.

Video cannot be sent raw

Start with a back-of-the-envelope calculation. Uncompressed Full HD video is 1920×1080 pixels, 3 bytes per pixel, 30 frames per second. Multiply it out and you get roughly 190 megabytes per second — a two-hour movie would exceed a terabyte. Even fiber connections cannot stream that. The only reason we watch video casually is that a video codec shrinks it by a factor of several hundred.

A codec is like a language. The sender (the encoder) and the receiver (the decoder) must share the same grammar before anything gets through. That is why an established language is so hard to displace — the grammar is baked into every phone and TV on the planet. A generation change is not just a technical upgrade; it means overturning that enormous web of installed compatibility.

The lineage: twenty years from H.264 to AV1

• H.264/AVC (2003): jointly standardized by ITU-T and MPEG. It spread into Blu-ray, broadcast TV, YouTube, and video conferencing, and more than twenty years later it remains the lowest common denominator that "plays everywhere."
• H.265/HEVC (2013): the designated successor, designed with the goal of roughly half the bitrate at the same quality — but its adoption stalled over the licensing mess we cover below.
• VP9 (2013): Google's royalty-free challenger, deployed at scale on YouTube.
• AV1 (2018): a royalty-free open format from the Alliance for Open Media (AOMedia), founded by Amazon, Google, Netflix, Microsoft, Mozilla, Cisco, Intel and others. It is VP9's successor and, in practice, the industry's collective answer to HEVC's licensing chaos.

Each generation is designed to deliver the same quality at a bitrate several tens of percent below its predecessor. For a streaming service, bitrate savings translate directly into lower delivery cost — which is exactly what makes those percentage points worth betting a generation change on.

Every generation shares the same skeleton

It may be surprising, but H.264 and AV1 have the same basic structure — the four-stage pipeline covered in Video Compression Basics:

1. Prediction: guess each block from nearby pixels (intra prediction) or from neighboring frames (inter prediction with motion compensation), and keep only the difference from reality
2. Transform: decompose that difference into frequency components with a DCT-family transform
3. Quantization: round away the components the human eye barely notices — this is where information is irreversibly discarded
4. Entropy coding: assign shorter codes to likelier values and pack everything tight

A generation change means adding more tools and finer-grained choices at every stage of this skeleton. It is not a revolution; it is accumulated refinement.

One equation drives every decision

More tools require a criterion for choosing among them. The criterion shared by every modern codec is rate–distortion optimization (RDO).

In plain words: pick whichever option minimizes the cost $J$, which balances how bad the picture looks against how many bits it takes. Here $D$ (distortion) is the error against the original image, $R$ (rate) is the number of bits the option costs, and $\lambda$ (lambda) is the exchange rate that says how much quality one bit is worth. For every candidate — block split, prediction mode, transform type — the encoder estimates $J$ and keeps the cheapest.

The fastest way to feel how quantization trades quality for bits is to watch it happen on a still image: