Students Professional
JA EN
Learn › Media in Production
·FREE·10 min read

HLS and DASH — How Video Actually Reaches You

Streamed video does not flow — a player downloads small files a few seconds long, one after another. What segments and manifests really are, the arithmetic behind how a player picks a rendition, where latency comes from, how low-latency modes cheat it, and why none of it works without a CDN.

#hls#dash#cmaf#abr#cdn#low-latency#streaming
ModalitytextTaskcompression

An analogy: a stack of chapters, not a river

Say "streaming video" and most people picture a tap: turn it on and a continuous flow arrives. What actually happens is far more mundane. A player is downloading small files off a web server, one at a time, in order.

Think of a part-work encyclopaedia sold volume by volume. A single sheet lists what exists; the reader consults it and goes back for the next volume. The shop just stacks books on a shelf — the reader decides which one to pick up and when. The list is the manifest; each volume is a segment holding a few seconds of video.

That one decision — cut the video into chapters — determines almost everything else. It is why quality can change mid-playback, why the files can sit in caches around the world, and why live streams end up tens of seconds behind reality.

Why video ended up travelling over HTTP

Streaming used to mean purpose-built protocols: RTMP from the Flash era, RTSP/RTP still visible in surveillance gear. These hold a connection open and let the server push whatever is happening right now.

In theory that is efficient. In operation it hit three walls: you needed special servers, you needed special ports that corporate firewalls tended to block, and — the decisive one — the traffic could not be cached. A stateful, per-connection stream looks like opaque bytes to anything in the middle, so there is no way to serve one copy to a thousand people who all want the same thing.

HLS (Apple, 2009) and MPEG-DASH (ISO/IEC 23009-1, 2012) inverted the model. Chop the video into ordinary files and serve them over ordinary HTTP. The server decides nothing; the client pulls what it needs, in the order it needs it. At that moment, twenty years of infrastructure built for the web — CDNs, caching, TLS, port 443 going through every firewall — became video delivery infrastructure. Streaming reached its current scale because of that switch at least as much as because of better codecs.

What a manifest and a segment actually look like

An HLS manifest is a text file, and it comes in two storeys: the multivariant playlist lists the available renditions, and each media playlist lists that rendition's segments.

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
v1080/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2"
v720/index.m3u8

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:1024
#EXT-X-MAP:URI="init.mp4"
#EXTINF:4.000,
1024.m4s
#EXTINF:4.000,
1025.m4s

A DASH manifest (the MPD) is XML, nested as Period (a stretch of time) → AdaptationSet (a group of interchangeable tracks: video, audio, subtitles) → Representation (one rendition). Rather than listing every segment, it can describe them with a template.

<AdaptationSet mimeType="video/mp4" segmentAlignment="true">
  <SegmentTemplate media="$RepresentationID$/$Number$.m4s"
                   initialization="$RepresentationID$/init.mp4"
                   duration="4" startNumber="1024" timescale="1"/>
  <Representation id="v1080" bandwidth="5000000" width="1920" height="1080" codecs="avc1.640028"/>
  <Representation id="v720"  bandwidth="2500000" width="1280" height="720"  codecs="avc1.64001f"/>
</AdaptationSet>

The differences are surface deep. One is text, the other XML. One plays natively on Apple devices, the other goes through the browser's Media Source Extensions. The media files underneath can now be identical for both. That is CMAF (ISO/IEC 23000-19): put the video in fMP4 and you publish two manifests over one set of segments, so storage and cache hold a single copy.

One prerequisite is easy to forget: segment boundaries must line up across every rendition. Boundaries fall on keyframes, so the encoder has to hold the GOP length fixed and place keyframes at identical positions in every resolution. The details belong to video compression from scratch, but a ladder whose keyframes drift will visibly jump on every switch.

What the player looks at when it picks a rendition

If the server decides nothing, then the player decides everything. That is ABR (adaptive bitrate), and its core is surprisingly plain addition and subtraction.

The player watches its buffer: how many seconds of video it holds but has not yet played. Write BkB_k for the buffer in seconds after fetching segment kk, dd for the segment duration, RkR_k for the bitrate of the chosen rendition, and CkC_k for the throughput actually achieved:

Bk+1=max⁡ ⁣(Bk−d RkCk,  0)+dB_{k+1} = \max\!\left(B_k - \frac{d\,R_k}{C_k},\; 0\right) + d
(1)

dRkd R_k is the size of one segment; dividing by CkC_k gives the seconds spent downloading it. Playback continues during the download, so the buffer drains by that amount, then gains dd seconds when the segment lands. The equation says only this: if the segment takes less time to fetch than it takes to watch, the buffer grows. The instant Rk>CkR_k > C_k, the buffer starts shrinking, and when it reaches zero playback stalls — that stall is the spinner.

So the naive rule for choosing the next rendition is:

Rk+1=max⁡{ R∈L  :  R≤γC^ }R_{k+1} = \max\{\,R \in \mathcal{L} \;:\; R \le \gamma \hat{C}\,\}
(2)

L\mathcal{L} is the set of available renditions, C^\hat{C} the throughput estimated from recent downloads, and γ\gamma a safety factor below 1. In words: take the best rung that still fits under your estimated throughput, discounted for safety. Push γ\gamma towards 1 and picture quality rises, but a slightly optimistic estimate now stalls playback. Real players fold in the buffer level as well, stepping down when it drains regardless of what the estimator claims.

Stepping down a rung means switching to the same picture encoded with coarser quantisation. It is worth seeing what that does to the image.

FIG 1As quality drops, fine detail disappears first, then at some point the image breaks into blocks. That is what a viewer sees when ABR drops a rung — spacing the rungs is partly the art of making the drop unnoticeable

How many rungs to build and how far apart is its own design problem, covered in designing a bitrate ladder.

Where the latency comes from

Moving to HTTP had a price, and the price is latency. The familiar complaint that a live stream runs half a minute behind broadcast falls almost straight out of the structure.

A player cannot request a segment until it exists as a finished file on the server. With four-second segments, the newest moment of video spends on average two seconds not yet being a file. On top of that, the player fills its buffer with several segments before starting playback so it does not stall immediately. With nn segments buffered:

L≈tenc+d+n d+tnetL \approx t_{\text{enc}} + d + n\,d + t_{\text{net}}
(3)

tenct_{\text{enc}} is encoding and packaging, dd is the wait for the segment to finish, ndnd is the pre-roll buffer, and tnett_{\text{net}} is the trip through the CDN. Most of the latency is segment duration times buffered count. Six-second segments times three is eighteen seconds before anything else counts, which is how naive setups land in the twenties or thirties.

The obvious fix — shorter segments — is not free. Every segment needs its own keyframe, so shortening them raises keyframe density and therefore the bitrate needed for the same quality, while request rates and manifest republishing both climb. Latency, quality and overhead pull against each other.

Going low-latency: ship it before it is finished

Both ecosystems reached the same idea: stop waiting for the segment to be complete, and send the front of it as it is produced.

DASH takes the direct route. A CMAF fMP4 segment can be written as a series of smaller chunks, so the packager streams them out with HTTP chunked transfer encoding while the segment is still being written. The player receives a four-second file in pieces, long before it exists in full.

HLS instead makes the pieces real, addressable objects. A four-second segment is split into parts of, say, half a second, published in the manifest as #EXT-X-PART. Two more mechanisms remove the round trips: a blocking playlist request (the player asks for the manifest as of a future part number and the server holds the response until that part exists) and #EXT-X-PRELOAD-HINT, which reveals the next URL before it is available.

There is a trap here that everyone walks into once. Low latency breaks throughput estimation. Normally you learn your bandwidth from "a four-second file arrived in half a second". Under chunked delivery the encoder, not the network, paces the transfer, so a four-second segment takes four seconds to arrive no matter how fat the connection is. A naive estimator concludes that throughput equals the current bitrate and never steps up. Low-latency-aware players measure the gaps between chunk arrivals precisely to escape this.

None of it works without a CDN

This is the payoff for moving to HTTP. A segment is an immutable file once written, so an edge cache can serve ten thousand viewers of the same programme from a single fetch to the origin.

Operationally it comes down to setting cache lifetimes separately. Long TTLs on segments, short TTLs on manifests. A live manifest changes every few seconds; cache it too long and every viewer stares at a stale index while the stream appears to freeze. Cut segment TTLs short and hit ratio collapses, sending the load straight back to the origin.

The second concern is collapsing requests. On a popular live event, thousands of clients hit the same new segment URL within the same instant. Unless the CDN coalesces those into one origin fetch, your own audience takes the origin down. For low-latency delivery, add two capability questions: can the edge hold a blocking playlist request open, and does it forward chunked responses as they arrive rather than buffering them whole? Buffering at the edge quietly undoes everything the packager did.

How this is used in practice

Who touches it, and when. The packaging engineer, when a new service's delivery configuration has to be decided. The live operations team, when someone asks for lower latency. The playback engineer, chasing a complaint that one device family cannot play the stream. All three argue over the same manifest, so knowing what the tags mean sets the pace of the conversation.

Producing the assets — the command you actually type

ffmpeg -i in.mp4 -c:v libx264 -b:v 2500k -g 96 -keyint_min 96 -sc_threshold 0 \
  -c:a aac -b:a 128k \
  -hls_time 4 -hls_playlist_type vod -hls_segment_type fmp4 \
  -hls_segment_filename 'v720/%d.m4s' v720/index.m3u8

The part that matters is -g 96 -keyint_min 96 -sc_threshold 0. At 24 fps, 96 frames is 4 seconds: the GOP is pinned to a divisor of the segment duration and automatic keyframes on scene changes are disabled. Drop those flags and segment boundaries drift apart between renditions, so the picture jumps on every switch. Production packaging usually runs through Shaka Packager or Bento4, where the constraint is identical.

Parameters you will be adjusting

Pitfalls that turn into incidents

Questions that come up in design review

"HLS or DASH?" — the standard answer is neither exclusively: share one CMAF asset set and publish both manifests. "Get latency down to five seconds" — shortening segments alone just spends quality and request budget, so you introduce parts or chunked transfer and at the same time revisit the player's pre-roll buffer and the CDN's support for it. "Why won't cache hit ratio improve?" — start by checking whether segment URLs carry a query parameter that changes per request. Codec selection itself is a separate argument, taken up in H.264 and AV1.

Summary

Comments

Sign in to comment