The Economics of GPU Cloud — Rent, Buy, or Commit

The same GPU carries four prices at once. Put on-demand, committed, spot, and owned hardware on a single equation: the break-even utilization, the true cost of spot once interruptions are priced in, the commitment burn rate, and the line items that never appear on the rate card.

The Same GPU Has Four Prices

"We don't have enough GPUs" sounds like an engineering problem. It's almost always an accounting one. It is rare that no silicon exists anywhere; what people usually mean is that at that price, the math doesn't work.

A single GPU model carries at least four prices simultaneously. On-demand, available right now at the highest hourly rate. Committed capacity, cheaper per hour because you promised a year or three of usage up front. Spot, dramatically cheaper because the provider can take it back whenever it wants it. And owned hardware, which you buy outright and then feed electricity yourself. The FLOPs are identical in all four cases. What differs is when you pay and who eats the loss when the capacity goes unused.

This article puts all four on one equation. No background required — it's multiplication and division the whole way down.

The Analogy: Taxi, Season Pass, Standby Seat, Own Car

Swap GPUs for transport and the four personalities fall out immediately.

• Taxi (on-demand): available the moment you want it, most expensive per kilometer
• Season pass (committed): pay up front, cheaper per trip, and the days you don't travel are simply gone
• Standby seat (spot): absurdly cheap, but you get bumped when the flight fills — and you keep whatever distance you already covered
• Own car (owned): a large payment on day one, then parking and insurance whether or not you drive

That last line is the whole game. Owned capacity costs you by the hour you hold it, not the hour you use it. So buy-versus-rent is never a performance argument; it's an argument about how much you drive. Getting that backwards — "I hear owning is cheaper" — is the single most common way teams lose money here.

Renting Is Just Multiplication

The rental side is honest and boring.

Put in words: the rental bill has exactly two ingredients, a price per hour and a number of hours. Read either one a fifth too low and the invoice arrives a fifth too high — nothing else is hiding in there.

Here $p$ is the price per GPU-hour and $H$ is the total GPU-hours you need. Rate times time. For training, $H$ is "number of GPUs × wall-clock hours" — eight cards for ten hours is 80 GPU-hours.

The hard part isn't $p$, it's $H$. Dividing required FLOPs by achievable throughput gives you a floor, and on top of that floor sit data stalls, communication waits, and every experiment you had to rerun. Adding GPUs shrinks wall-clock time but inflates $H$ through communication overhead — that scaling behavior is covered in distributed training. Budget misses almost always come from the time side, not the rate side.

Owning Is Division by Utilization

Owned hardware inverts the structure. The purchase price leaves your account whether the card computes or idles, so you end up dividing one fixed sum by the hours it actually worked. The smaller the divisor, the higher the hourly cost.

Fix some symbols. $K$ is the all-in acquisition cost per GPU — not just the card, but its share of the chassis, networking, and electrical work. $L$ is the depreciation life and $T = 8760L$ is the wall-clock hours you hold it. $M$ is total operations cost (staff, spares) over that life, $P$ is GPU power draw in kW, $\gamma$ is facility PUE, and $e$ is the electricity rate per kWh. Finally $u$ is utilization — the fraction of held hours spent actually computing.

Said in words: pile up everything you spent to buy and babysit the card, spread it evenly across the hours it actually computed, then add the electricity you burn only while it is computing.

The first term is "what it costs to own, divided by the hours it worked." The second is "what it costs to run, charged only while running." PUE is total facility power divided by IT equipment power, so it is always at least 1: cooling and conversion mean the wall draws more than the GPU's nameplate (the chip power budget).