Designing a Bitrate Ladder — The First Job You Get in Streaming

Why one video needs several renditions. From how ABR works to rate-distortion curves per resolution and the convex hull that picks your rungs, the three ways excess bitrate costs you, how PSNR, SSIM and VMAF differ and which one decides, and per-title encoding — ending in a step-by-step procedure from choosing clips to keeping the ladder honest.

An analogy: a shop that sells one size

A clothes shop that makes one size sells almost everyone something either too tight or too baggy. Stock every centimetre instead and inventory costs swallow the business. How many sizes, and how far apart is the design problem, and it is a commercial one as much as a technical one.

In streaming, that size range is the bitrate ladder. Viewers arrive on everything from fibre to a congested mobile cell, and a single viewer's connection changes minute to minute. Ship one rendition and you are choosing between "too fat, so it stalls" and "too thin, so it looks bad". Add rungs and both encoding cost and storage grow with the count.

Designing that ladder is usually the first real task handed to someone joining a streaming team. It looks unglamorous, and it is one of the very few places where picture quality, delivery cost and viewer experience all intersect at one number.

ABR — the player chooses for itself

Essentially all modern streaming is ABR (adaptive bitrate). The mechanism is simple: encode the same content at several qualities and cut each into segments a few seconds long. List the rungs in a manifest (.m3u8 for HLS, .mpd for DASH) and the player, watching its measured throughput and buffer level, re-picks a rung for every segment.

The server decides nothing. It lays the rungs out; the client chooses. So the designer's entire job is deciding which rungs to lay out.

For a player to switch mid-stream, segment boundaries have to line up across every rendition, which means keyframe positions must match at every resolution. That is GOP design, covered in video compression from scratch, so this article takes it as given. Just remember that in a ladder whose keyframes are not aligned, every extra rung is another chance for the picture to jump on a switch.

Bitrate and quality are not proportional

Before picking rungs, one property has to be internalised: doubling the bitrate does not double the quality.

The curve relating coded quality to rate is the rate-distortion (R-D) curve, and it is concave. Down at low rates, a few extra bits buy a large jump in quality; up at high rates, the same bits barely move the needle. It is the same behaviour seen in JPEG quantization: the finer the coefficients you keep, the less each bit returns.

Decide "let's go high to be safe" without knowing the shape of that curve and you pay forever for quality nobody can see. Know where the knee sits and you can buy the same perceived quality at the lowest rate that delivers it. Ladder design is, at bottom, the question of which points on this curve to pick.

The curves cross when resolution changes

A rung is not just a bitrate — it is a bitrate and a resolution. And the same bitrate gives different quality depending on which resolution you encode at.

Send a high resolution at a low bitrate and there are not enough bits per pixel: quantization coarsens and you get blocking and mush. Spend those same bits on a lower resolution and the encode is clean at that resolution; upscaling on playback then looks better than the higher resolution did. At high rates the relationship inverts — the low resolution saturates (it cannot invent detail it never had) and the higher resolution overtakes it.

So each resolution has its own R-D curve, and the low resolutions sit on top at low rates while the high resolutions sit on top at high rates, crossing somewhere in between. At a given bitrate $R$, the resolution to use is

where $Q(r, R)$ is the quality obtained by encoding at resolution $r$ and bitrate $R$, and $\arg\max$ picks the $r$ that maximises it. Put in words: at this bitrate, encode the clip at every candidate resolution and keep whichever one comes out looking best. Which one wins changes as the bitrate changes, and that is precisely why the curves cross.

The envelope traced by whichever curve is highest at each rate is the convex hull, and the rule is that rungs are chosen from that hull.