From H.264 to AV1 — What a Codec Generation Change Really Involves

H.264 has ruled video for two decades; AV1 arrived promising royalty-free efficiency. We unpack the transition along three axes — coding efficiency, compute cost, and licensing — and end with a practical framework for deciding which codec to ship, starting from zero background.

Video cannot be sent raw

Start with a back-of-the-envelope calculation. Uncompressed Full HD video is 1920×1080 pixels, 3 bytes per pixel, 30 frames per second. Multiply it out and you get roughly 190 megabytes per second — a two-hour movie would exceed a terabyte. Even fiber connections cannot stream that. The only reason we watch video casually is that a video codec shrinks it by a factor of several hundred.

A codec is like a language. The sender (the encoder) and the receiver (the decoder) must share the same grammar before anything gets through. That is why an established language is so hard to displace — the grammar is baked into every phone and TV on the planet. A generation change is not just a technical upgrade; it means overturning that enormous web of installed compatibility.

The lineage: twenty years from H.264 to AV1

• H.264/AVC (2003): jointly standardized by ITU-T and MPEG. It spread into Blu-ray, broadcast TV, YouTube, and video conferencing, and more than twenty years later it remains the lowest common denominator that "plays everywhere."
• H.265/HEVC (2013): the designated successor, designed with the goal of roughly half the bitrate at the same quality — but its adoption stalled over the licensing mess we cover below.
• VP9 (2013): Google's royalty-free challenger, deployed at scale on YouTube.
• AV1 (2018): a royalty-free open format from the Alliance for Open Media (AOMedia), founded by Amazon, Google, Netflix, Microsoft, Mozilla, Cisco, Intel and others. It is VP9's successor and, in practice, the industry's collective answer to HEVC's licensing chaos.

Each generation is designed to deliver the same quality at a bitrate several tens of percent below its predecessor. For a streaming service, bitrate savings translate directly into lower delivery cost — which is exactly what makes those percentage points worth betting a generation change on.

Every generation shares the same skeleton

It may be surprising, but H.264 and AV1 have the same basic structure — the four-stage pipeline covered in Video Compression Basics:

1. Prediction: guess each block from nearby pixels (intra prediction) or from neighboring frames (inter prediction with motion compensation), and keep only the difference from reality
2. Transform: decompose that difference into frequency components with a DCT-family transform
3. Quantization: round away the components the human eye barely notices — this is where information is irreversibly discarded
4. Entropy coding: assign shorter codes to likelier values and pack everything tight

A generation change means adding more tools and finer-grained choices at every stage of this skeleton. It is not a revolution; it is accumulated refinement.

One equation drives every decision

More tools require a criterion for choosing among them. The criterion shared by every modern codec is rate–distortion optimization (RDO).

In plain words: pick whichever option minimizes the cost $J$, which balances how bad the picture looks against how many bits it takes. Here $D$ (distortion) is the error against the original image, $R$ (rate) is the number of bits the option costs, and $\lambda$ (lambda) is the exchange rate that says how much quality one bit is worth. For every candidate — block split, prediction mode, transform type — the encoder estimates $J$ and keeps the cheapest.

The fastest way to feel how quantization trades quality for bits is to watch it happen on a still image:

What AV1 actually added

Here are the headline additions relative to H.264.

Freedom in block partitioning. H.264's basic unit was the 16×16 "macroblock." AV1 starts from superblocks up to 128×128 and recursively splits down to 4×4 in a variety of shapes. Flat regions like sky or walls get covered by huge blocks in one stroke, while busy contours get fine splits — and that allocation pays off.

Sheer volume of prediction modes. Where H.264 offered 9 directional intra modes on 4×4 blocks, AV1 has 56 directional modes alone. Inter prediction gained warped motion compensation, which approximates rotation and zoom, and compound prediction, which blends multiple predictors.

A new generation of in-loop filters. On top of the classic deblocking filter, AV1 adds CDEF to suppress ringing and a loop-restoration filter to recover lost detail. It also introduces film grain synthesis — a genuine change of mindset: since noise is compression's worst enemy, remove it, transmit only its statistics, and re-synthesize it on the player's side.

Entropy coding. H.264's CABAC arithmetic-codes one binary decision at a time; AV1 arithmetic-codes multi-symbol alphabets directly, with probability tables adapted per context. The underlying theory is exactly what we covered in Entropy Coding: make the likely things short, relentlessly.

The price of efficiency is paid in compute

The bill for all those extra tools arrives as search cost. RDO works by estimating candidates and keeping the minimum, so as the options multiply — partition shapes × prediction modes × transform types — the combinations grow multiplicatively.

Indeed, AV1's reference encoder was impractically slow when first released. Optimized implementations such as SVT-AV1, led by Intel and Netflix, plus smart search pruning, have since brought it into practical range for streaming work. And note one crucial asymmetry: encoding is heavy, but decoding is far lighter. When you compress once and play back millions of times, even an expensive encoder pays for itself.

Comparing them in code

With ffmpeg you can try both generations through the same front door:

# H.264 (x264): the veteran with twenty years of optimization behind it
ffmpeg -i in.mp4 -c:v libx264 -crf 23 -preset medium out_h264.mp4

# AV1 (SVT-AV1): lower preset numbers = slower and more efficient
ffmpeg -i in.mp4 -c:v libsvtav1 -crf 32 -preset 6 out_av1.mp4

One warning: -crf values are not comparable across codecs. x264's crf 23 and SVT-AV1's crf 32 merely tend to land near similar quality — they are different scales. To compare at matched quality, check with your eyes plus metrics like VMAF or SSIM.

Licensing — the other reason for the changeover

H.264's patents were consolidated into the MPEG LA (now Via-LA) pool, with clear rates and caps. Crucially, the pool announced it would never charge royalties for internet video that is free to end users, which cleared the way for the web's mass adoption.

HEVC stumbled exactly here. Its patent holders fractured into multiple pools plus independent licensors, and for years nobody could predict what the total bill would be. The big tech companies that hated this uncertainty founded AOMedia and built AV1 as a royalty-free format whose members cross-license their patents at no charge — that is the political half of the story.

That said, "royalty-free" is not "litigation-proof." Patent pools claiming to represent non-AOMedia rights holders (Sisvel among them) offer AV1 licenses, and the merits of those claims are disputed. Standard practice is to run a legal review before any large-scale rollout.

The adoption decision: what actually gets weighed

Technical superiority alone does not settle it. Three axes do.

1. Your viewers' device mix. Software-decoding AV1 hits older devices hard in CPU and battery. What share of your audience has hardware AV1 decoders?
2. Encoding cost versus delivery cost. The more views a title gets, the sooner bitrate savings outrun the encoding bill. The standard playbook is staged migration: convert popular titles to AV1 first, leave the long tail on H.264.
3. Licensing tolerance. Can you absorb HEVC's royalty negotiations, or do you prioritize royalty-free?

The major streaming services do not flip a switch either: they keep the same title in both H.264 and AV1 and serve whichever the device can handle. A generation change is not a replacement — it is a long period of running side by side.

How this is used on the job

When a video encoding engineer at a streaming service specifies the delivery settings for a new title, this article is essentially the checklist. The knobs are -crf (quality), -preset / -cpu-used (the speed-versus-efficiency dial), and -g (keyframe interval), with VMAF as the reference metric for quality checks. Designing the resolution-by-bitrate grid is the job of the bitrate ladder.

Three pitfalls. First, re-encoding compounds losses — transcoding an H.264 file to AV1 cannot recover what H.264 already threw away, so always start from the highest-quality master available. Second, never carry crf values across codecs; most "same crf, weird quality" tickets trace back to this. Third, shipping AV1 to devices without hardware decode: the video may play, but heat and battery drain will wreck the experience unless you keep an H.264 fallback.

If an interviewer asks "why is H.264 still everywhere?", a solid answer pairs its universal compatibility, light encoding, and predictable licensing against AV1's efficiency advantage.

Read next

• Video Compression Basics — the four-stage pipeline of prediction, transform, quantization, and coding, step by step
• Entropy Coding — the theory behind "make the likely things short"
• The Bitrate Ladder — the design task that comes right after codec selection

Takeaways

• H.264 and AV1 share one skeleton. The real substance of a generation change is the sheer volume of tools at each stage, plus the RD optimization that chooses among them
• Efficiency is paid for in encoder compute. The compress-once, play-millions economics of streaming is what makes it worthwhile
• AV1's birth was half technology, half backlash against HEVC's licensing chaos
• The adoption call weighs device mix × view counts × licensing tolerance — and in practice the codecs run side by side rather than replacing each other