Advanced Packaging — How CoWoS and HBM Stacking Became the Bottleneck for AI

What limited the supply of AI accelerators was not leading-edge logic wafers but the step that puts several finished chips into one part. From the reticle limit and the exponential math of yield, through 2.5D interposers, HBM stacks and TSVs, to hybrid bonding and the bill that heat and warpage send back.

The Bottleneck Is No Longer the Transistor

The shortage of AI accelerators has been with us for a while. The intuitive explanation is that the leading-edge 3nm and 5nm lines are full. But the bottleneck reported again and again sat somewhere else: after the transistors are patterned, in the step that takes several finished chips and assembles them into a single part. Packaging capacity, and the supply of the HBM that goes into it, were setting the shipment numbers.

For a long time a package was a box that protected a chip and brought its pins out. Performance was decided inside the die; the outside was an afterthought. That inverted over the past few years. Today, how you build the wiring outside the die decides bandwidth, power and price. Semiconductor coverage gravitates toward node numbers, but what determines the accelerator in front of you is often what surrounds the die.

Why did it come to this? It starts with a piece of physics: you cannot make one die as large as you want. How a single die gets made is covered in How Chips Are Made. This article picks up where that one ends — splitting, placing, stacking, connecting.

An Analogy: Separate Buildings, or a Skybridge

Picture two departments that exchange a mountain of paperwork every day. There are three ways to seat them.

• Same floor: hand it to the desk next door. Fast and cheap, but a floor only holds so many people
• Different buildings: room for everyone, but paperwork now crosses a public street on a cart. Slow, and it costs staff
• Next door, joined by a skybridge: never touches the street; a few dozen steps

That third option is 2.5D integration. Dies are connected inside the package by wires that are extremely short and extremely numerous. The skybridge is a slab called an interposer, and TSMC's version of this approach is CoWoS (Chip on Wafer on Substrate).

There is a level above that. Stacking the departments on top of each other is 3D integration. Punching through the floor is shorter still than a skybridge — but the upper floor takes the heat of the one below it. Nearly every constraint in this article maps back to some part of this picture.

There Are Two Ceilings

Naively, the fastest thing to build is one giant die with everything on it. Wires stay short, and there are no seams in between. Designers would love to. What stops them is not a performance argument but manufacturing physics, and it comes in two ceilings.

The first is the reticle limit. A scanner can only print so much area in a single exposure — roughly 26 mm × 33 mm on current tools, about 858 mm². A circuit larger than that cannot exist as one printed pattern. No amount of wanting moves that line.

The second is yield. Defects land on a wafer at random, so the larger a die is, the lower its chance of stepping on none of them. And the fall-off is not proportional — it is exponential. Double the area and the yield is not halved, it is squared. Where the reticle limit decides what cannot be built, this decides what can be built but never pays for itself. In practice the second ceiling usually bites first.

Because of these two, designers commit up front to splitting the design and sewing it back together later. And the performance of what they sewed is decided almost entirely by how they sewed it — by the package. That is why the wiring conversation now comes before the arithmetic-unit conversation.

The Yield Arithmetic: Why Splitting Pays

If defects fall randomly across the wafer, the probability that a die of area $A$ steps on none of them is

$Y$ is yield (the fraction of good dies), $A$ is the area of one die, and $D_0$ is defects per unit area. Read it as: make the die bigger and the good fraction decays exponentially.

Put in words, it says: the wider you spread a die, the faster its chance of dodging every defect thins out. Doubling the area squares the yield rather than halving it precisely because area sits in an exponent here, not in a multiplication.