From TCP to QUIC — Reinventing Reliable Communication

The internet is built on a foundation that makes no promise of delivery. This piece starts from zero: how TCP manufactures reliability out of sequencing, retransmission and congestion control, why HTTP/2 hit a wall, and why QUIC deliberately rebuilt all of it on top of UDP.

Reliability Is Something You Manufacture

The internet is not reliable because someone laid down sturdy cable. It is reliable because we stack a paperwork protocol on top of a foundation that is assumed to break.

Here is the metaphor. You want to send a friend a long letter, but the only thing you can use is a postcard — write the address, drop it in the box. Postcards usually arrive, but nothing is guaranteed. Some get lost. Some arrive out of order. Some arrive twice. There is no complaints desk.

Under those constraints there is exactly one way to deliver the whole letter, in order, for certain: number the postcards, have the recipient write back "I have everything through number 32," and resend whatever goes unacknowledged. That is the entirety of what TCP does. Not deep theory — clerical procedure layered on top of an unreliable substrate.

The Foundation, IP, Only Carries Postcards

At the bottom sits IP, which passes small chunks called packets from router to router. Each router forwards and forgets; nobody tracks whether anything arrived.

The crucial part: a congested router dropping packets is not a malfunction, it is normal operation. Inside a router is a queue of packets waiting to go out. When traffic arrives faster than it leaves, the queue grows, and the overflow hits the floor. And there is generally no mechanism to tell the sender it happened.

Every constraint in TCP's design flows from this. The sender can only infer loss and congestion from the absence of a reply.

Job One — Sequencing and Acknowledgement

TCP presents itself to the application as an unbroken stream of bytes. Whatever you write() comes out of the peer's read() in order, with nothing missing and nothing duplicated. This is the byte-stream abstraction.

The mechanism is unglamorous: every byte carries a sequence number, and the receiver replies with an ACK naming the position through which it has received data contiguously. This is a cumulative ACK — saying "through 32" also confirms everything before it, which keeps acknowledgement traffic small.

The price shows up later. A cumulative ACK can only describe the contiguous prefix, so it has no way to say what has arrived beyond a hole. SACK (selective acknowledgement, RFC 2018) was bolted on afterwards precisely to fill that expressive gap.

Job Two — Retransmission, or Knowing When to Give Up

No reply came. How long do you wait before resending? Too short and your redundant retransmissions make the congestion worse; too long and the connection appears frozen. TCP continuously measures round-trip time (RTT) and derives a timeout (RTO) from its average and its variability (RFC 6298).

Here $R$ is the round-trip time just measured, $\mathrm{SRTT}$ is a running average of it (one that forgets the past gradually), and $\mathrm{RTTVAR}$ is how much that time jitters. All the formula says is: wait for the usual round trip plus four helpings of jitter, and if nothing comes back, call it lost. It is the common-sense rule — be more patient on a jittery link — written down.

Waiting on a timer is slow, though, so TCP also has fast retransmit. A receiver sitting on a hole keeps replying "still only through 32" as later data arrives, and once the sender sees the same ACK three times over, it resends without waiting for the timer.

And here a problem appears that will matter later. When an ACK comes back for a retransmitted packet, the sender cannot tell whether it answers the original transmission or the retransmission. Sequence numbers denote a position in the data, so a resend puts the same number on the wire twice. Since the two are indistinguishable, that RTT sample has to be thrown away (Karn's algorithm) — meaning measurement quality degrades exactly when the network is at its worst.

Job Three — Congestion Control, Guessing at Invisible Traffic

This is the hard part. Nothing tells you how congested the path is. TCP took a gamble: treat packet loss as a proxy signal for congestion.

The sender maintains an allowance of data it may put in flight without waiting for replies — the congestion window (cwnd). A new connection begins in slow start, doubling cwnd every round trip to feel for the ceiling. Once losses appear it switches to congestion avoidance, governed by AIMD (additive increase, multiplicative decrease).

$w$ is the congestion window. In plain terms: while things are going well it takes a full round trip to gain one unit, and the instant a loss appears it gives up a large fraction at once. Up the stairs, down the slide. The asymmetry is deliberate. If both directions were multiplicative, the flow already taking the most would keep winning and the system would never settle. Making increase additive and decrease multiplicative means the greediest flow loses the most on every loss event, so competing connections converge toward an even share. The asymmetry buys fairness.

And throughput comes down to a single line.

That is: the amount you can put in flight per round trip, divided by how long a round trip takes. Because RTT sits in the denominator, distance alone makes you slow. Buying a fatter pipe does not shorten the trip to the other side of the planet.

Treating loss as the signal has a side effect. When router buffers are deep, TCP keeps pushing — nothing has been dropped, so there must be room — and the queue simply grows without producing loss. Throughput looks fine while round-trip time swells into the hundreds of milliseconds. This is bufferbloat. CUBIC (the default on many Linux distributions) reshapes how the window grows; BBR instead tries to sidestep the trap altogether by estimating bandwidth and round-trip time directly rather than waiting for loss.

The Wall HTTP/2 Hit — Head-of-Line Blocking

Under HTTP/1.1 a connection handled one request at a time, so browsers opened roughly six parallel connections to the same server. HTTP/2 tidied this into a single connection carrying multiplexed streams. Fewer connections, compressed headers — on paper, a clear improvement.

Yet on lossy links it can come out slower than HTTP/1.1, and the cause is TCP's own promise. TCP guarantees an in-order byte stream across the whole connection, so when one segment is lost, data that has already arrived behind it sits in the kernel's receive buffer instead of reaching the application — even when it belongs to a completely different stream. The whole river is dammed until the hole is filled. That is head-of-line blocking.

Spread over six connections, one stall left five flowing. Consolidating onto one connection meant a single loss now stops every stream.

This is not a design error in HTTP/2. The cause is TCP's property that the unit of ordering is the entire connection, and since its API is a byte stream, no layer above can fix it. Fixing it required rebuilding the transport layer itself.

Why Rebuild on UDP, of All Things

If you are writing a new transport, the natural move is to define one alongside TCP directly on IP. QUIC did not, and the two reasons are less about engineering than about reality.

Reason 1: TCP can no longer be changed (ossification). Paths are full of middleboxes — NATs, firewalls, WAN optimizers — that inspect TCP headers to decide what to do. Packets carrying unfamiliar options are sometimes silently discarded. So a new TCP feature can be specified and still not traverse the network. That is part of why TCP Fast Open (RFC 7413) and Multipath TCP never saw the deployment their designs deserved.

Reason 2: TCP lives inside the OS kernel. Shipping an improvement means waiting for the world's devices to update their operating systems — years, and for some hardware, never. Userspace, on the other hand, ships with the application, so a new transport can ride along with every release. And the only pipe that userspace can drive and that still traverses arbitrary paths is UDP.

UDP is essentially IP with a port number attached and nothing else: no ordering, no retransmission. That emptiness is what keeps middleboxes from interfering. QUIC (RFC 9000, 2021) implements everything TCP did — plus things TCP could not — inside that empty pipe. It began as a Google experiment and was standardized at the IETF.

Five Things QUIC Rebuilt

1. Connection setup and encryption merged. Previously TCP's three-way handshake cost a round trip, and TLS key exchange stacked on top of it; even with TLS 1.3 that is two round trips before application data flows. QUIC folds transport establishment and the TLS 1.3 key exchange into one procedure: one round trip on first contact, and 0-RTT on a return visit, carrying request data in the very first packet. Because 0-RTT data can be replayed wholesale by an attacker, though, it is only safe for requests with no side effects.

2. Streams became first-class in the transport. The unit of ordering is the stream, not the connection. If a packet belonging to stream A is lost, data that arrived for stream B is still delivered to the application. HTTP/2's problem disappears here. To be honest about it, it does not vanish entirely: when HTTP/3's header compression (QPACK) references its dynamic table, that portion regains an ordering dependency.

3. Packet numbers are never reused. Separately from the data's position, QUIC stamps every transmitted packet with a monotonically increasing number. Retransmitted data goes out in a packet with a fresh number, so every ACK unambiguously answers exactly one transmission. RTT samples never have to be discarded, and loss detection becomes crisp (RFC 9002 declares loss once a packet is three numbers behind an acknowledged one, or once $9/8$ of $\max(\mathrm{SRTT},\ \text{latest RTT})$ has elapsed). This is Karn's problem solved at the root.

4. Connection IDs. A TCP connection is identified by the tuple of source IP, source port, destination IP and destination port — so moving from Wi-Fi to cellular changes your IP and kills the connection. QUIC gives each connection a connection ID independent of the addresses, so the same connection continues across an IP change (connection migration).

5. Even headers are encrypted. QUIC encrypts most of the header, not just the payload, specifically to stop middleboxes from reading along and making their own decisions. It is a deliberate defence against being ossified again — the lesson of TCP's paralysis written straight into the specification.

Where to Put Your Hands

ss -ti                                   # cwnd, rtt and retransmit counts, right now
sysctl net.ipv4.tcp_congestion_control   # cubic or bbr?
curl --http3 -sv https://example.com/    # try the request over HTTP/3
sysctl -w net.core.rmem_max=7500000      # default UDP receive buffers are often too small for QUIC
# Kill the classic tens-of-milliseconds stall on small writes:
#   s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

How This Shows Up on the Job

Who touches it, and when. Backend engineers, SREs, CDN operators and mobile developers, at the moment someone says "it's slow for some reason." The goal is almost always the same: decide whether the slowness is bandwidth, round-trip time, or loss.

What to look at. For TCP, ss -ti is the shortest path — it reports cwnd, rtt and retrans together. Lots of retransmits means loss; a cwnd that stays small means congestion control is holding you back; a large rtt means distance or queueing. For QUIC the payload is encrypted, so tcpdump will not show you anything useful; instead take the qlog your library emits and load it into qvis to see per-stream flow and loss. Not knowing that "QUIC is invisible in Wireshark" is the single most common way to burn an afternoon during a migration.

Parameters you will actually type. net.ipv4.tcp_congestion_control (cubic / bbr), net.core.rmem_max and wmem_max, TCP_NODELAY, TCP keepalive. To get clients actually using HTTP/3 you must advertise h3 — via the server's Alt-Svc header or a DNS HTTPS record (RFC 9460). Enabling it on the server alone will not make clients use it.

Pitfalls that turn into incidents.

1. Nagle's algorithm meeting delayed ACK. Split a small message across two write() calls and the sender holds the second piece waiting for an ACK while the receiver delays the ACK because it has nothing to send back. Both wait, and you stall for tens of milliseconds (about 40 ms on Linux). For RPC and games that exchange small messages, TCP_NODELAY — or coalescing into one write() — is the standard fix.
2. Buying bandwidth does not shrink RTT. Long-haul transfers do not speed up with a fatter pipe. What helps is shortening the distance itself, via a CDN or edge placement.
3. Bufferbloat. When speed tests look great but interaction feels sluggish, a queue has formed somewhere on the path. Loss-based controllers like CUBIC cannot detect it; this is where BBR or fq_codel earn their keep.
4. QUIC costs CPU. TCP offloads much of the receive path to the kernel and the NIC, while QUIC decrypts and processes packet by packet in userspace. On busy servers, verify that UDP GSO/GRO is in play.
5. Networks that block UDP genuinely exist. Corporate networks and some public Wi-Fi block or rate-limit UDP. Always keep HTTP/3 able to fall back to TCP. Never build an h3-only deployment.
6. Do not use 0-RTT for non-idempotent requests. Since the data can be replayed, sending something like "execute this payment" over 0-RTT opens a window for double execution.

How it gets asked in interviews and design reviews. "Why didn't HTTP/2 make things faster?" "How does QUIC's retransmission differ from TCP's?" "Why is QUIC on UDP?" All three reduce to the same two axes: where the unit of ordering lives, and who is able to ship the code to the world. Hold those, and you can reason your way through without memorizing the specifications.

Summary

• IP carries nothing but postcards that usually arrive. Reliability is manufactured afterwards out of numbering, acknowledgement and retransmission
• TCP's congestion control is a bet that loss stands in for congestion; additive increase with multiplicative decrease is what buys fairness
• HTTP/2 stalled because the unit of ordering is the whole connection — unfixable from any layer above
• QUIC chose UDP not for performance but because TCP had become unchangeable, pinned down by middleboxes and by the kernel
• What that bought: per-stream ordering, one-round-trip (or zero, on return) setup, connection IDs that survive an IP change, and unambiguous loss detection

There is an opposite strategy to resending what was lost: mix in redundancy up front so the receiver can reconstruct on its own. That approach, which wins wherever you cannot afford to wait for a round trip, is covered in Error Correction from Scratch — Sending Data on the Assumption It Will Break. And to see how the round-trip and buffering effects here surface as visible latency in video delivery, read HLS and DASH — How Video Actually Reaches You.