Serverless and Cost Design — Cloud That Won't Bankrupt You

"Pay for what you use" also means "pay for what someone else uses of you." Starting from zero, this piece builds up scale-to-zero, the GB-second billing unit, and the accident patterns that make a bill grow exponentially — ending with a kill switch you can actually design into your own project.

What "Pay for What You Use" Really Means

For most of computing history, renting a server meant renting a room. You paid monthly for the space, got a key, and put your belongings — your program — inside. Nobody visits at 3 a.m., the office is shut over the holidays, and the rent is exactly the same.

Serverless turned that lease into a taxi ride. The meter reads zero while nobody is in the car, and starts the moment someone gets in. Through a night when no request arrives, your app costs literally nothing. That is scale to zero: when nothing is being used, the fleet shrinks to zero instances.

So far this sounds like pure upside. But flip the same property over and it reads differently: the person deciding your bill is not you, it is whoever sends the requests. With a rented room, a crowd showing up doesn't change the rent (the room simply overflows). A taxi keeps the meter running for as long as someone keeps naming destinations. This article is about keeping the upside while designing a way to stop that meter on purpose.

There Are Still Servers

One misconception to clear first: serverless still runs on servers. What disappeared is not the machine — it is the time you hold the machine reserved.

The provider keeps a large fleet and multiplexes many customers' functions across it. When a request arrives, a small execution environment holding your code (a container, or a lighter isolated sandbox) starts up, runs, stops when the work is done, and is discarded if nothing calls it for a while. That start-use-discard cycle is, in operating-system terms, exactly the creation of a process and its address space — what gets built and what gets protected is covered in Processes and Memory from Scratch — What Is the OS Actually Protecting?.

Two practical consequences follow. The first is the cold start. Being called from a discarded state means paying for environment startup, runtime initialisation, and loading your code, so the first response is slower. Call it again immediately and the environment is still alive, so it's fast. The same function's latency varies by a large factor depending purely on how it happens to be called.

The second is statelessness. Neither a file you wrote to local disk nor a value you stashed in a global variable is guaranteed to survive to the next invocation. The nasty part isn't that state sometimes vanishes — it's that state often persists. Warm environments get reused, so your cache appears to work throughout testing, then falls apart the moment production scales out and requests scatter across separate environments. The only safe style is code that is correct whether the state is there or not.

What Exactly Is Being Counted

The shortest path to competence here is learning to read the invoice. Billing for function execution is essentially a sum of three products.

In words: the price of being called, plus the price of memory × time, plus the price of data leaving the cloud. $N$ is the number of invocations and $p_r$ the price per invocation. $m_i$ is the memory (in GB) allocated to the $i$-th execution and $t_i$ its duration in seconds; that sum of products is the unit known as GB-seconds. $p_c$ is the price per GB-second, $E$ is the volume of traffic that left the provider's network, and $p_e$ its price. The rates themselves vary by provider, region and year, so memorising them is pointless — but what is being counted does not change.

Renting one always-on server looks like this instead:

which says simply hourly rate × hours in a month (730 ≈ 24 × 365 ÷ 12). Notice that $N$ does not appear. Zero calls or a million calls, the number is the same — though in the second case the box falls over.

Put the two side by side and you get one clean decision axis.

which says the break-even point is the fixed cost divided by the per-invocation cost. Below that many calls per month, serverless is cheaper; above it, the fixed box wins. The intuition behind the algebra: serverless has a high unit price but a floor of zero, while an always-on server has a low unit price but a high floor. So serverless wins while traffic is small, and wins even harder when the ratio between peak and trough is large; a workload that runs hot twenty-four hours a day is cheaper on fixed capacity. The accurate slogan is not "serverless is cheap" — it is "serverless is cheap when the troughs are deep".

More Memory Can Cost Less

The GB-second unit hides one counter-intuitive behaviour. On most platforms, the CPU share you get scales with the memory you allocate. Double the memory setting and CPU-bound work finishes in roughly half the time. If $m$ doubles while $t$ halves, the product $mt$ is unchanged — you got twice the speed for the same money.

There is a condition, though. A function that spends most of its wall clock waiting on an external API or a database won't see $t$ shrink at all. There, only $m$ doubles, and so does the bill. Memory is therefore not a "set it high to be safe" or "set it low to save money" knob — it is a parameter you decide by asking whether the function is CPU-bound or I/O-bound. The method is simple: measure a representative input at several memory settings and pick the point where $m \times t$ is smallest.

Bankruptcy Comes from the Tail, Not the Average

What makes usage-based billing dangerous is not the average — it is that there is no ceiling by default. And when a bill explodes, it usually explodes multiplicatively rather than additively.

The classic accident is a function that ends up invoking itself. You write "when an image lands in the bucket, generate a thumbnail" and then write the thumbnail back into the same bucket. The output fires the trigger again, producing more output. If one execution spawns $b$ executions on average, the count at depth $n$ is

$$N_n = b^{\,n}$$

which is to say it doubles and redoubles: it grows without bound whenever $b > 1$, and dies out when $b < 1$. That difference is a single branch in your code, and the outcomes are separated by roughly the gap between a few dollars and a few thousand overnight.

Structurally similar accidents are everywhere. A retry snowball, where failed executions are retried automatically and the extra load causes more failures. A loop that calls a metered external API — LLM inference, say — with an input batch a hundred times larger than anticipated. An unauthenticated public endpoint being hammered by crawlers or attackers. And, more often than people expect, egress: serve large files directly without a CDN in front and data transfer, not compute, becomes the top line item.

The danger of exponential growth is how flat it looks on the way up. Linear cost shows up proportionally on your monitoring graph, so you notice it. Exponential cost takes very little time to become visible, and by the time it is visible you are already past the point of prevention. For a feel for how much difference an order of growth makes in real measurements, see When Big-O and Your Benchmarks Disagree.

Build the Stop Before You Build the Start

One principle covers all of this: don't design for "notice it and stop it" — design so it stops by itself. The reason is simply that billing data is not real time. Usage has to be aggregated before a budget alert fires, and that takes a while. Treat a budget alert as a smoke detector, not an invoice: when it goes off, nothing has been stopped. So build the stopping mechanism in three separate layers.

Squeeze the entrance. The highest-leverage control is a concurrency limit. Cap the number of simultaneous executions and, no matter how hard the endpoint is hit, the number running at once is bounded — which puts a ceiling on your spend per unit time. In an emergency, setting that cap to zero shuts the function down completely without a redeploy (reserved concurrency on AWS; maximum instances on Cloud Run / Cloud Functions). Require authentication on anything that doesn't need to be public, and put rate limiting and a WAF in front of anything that does.

Squeeze each execution. Don't leave the timeout at the maximum default — you are billed for time spent waiting on an external API too. Bound the retry count and maximum backoff, and route messages that still fail into a dead-letter queue so the retry loop is broken rather than repeated forever.

Squeeze the whole account. Don't let a budget alert end as a notification; wire it so the notification actually stops the billing. On GCP the standard shape is "budget alert → Pub/Sub → a function that unlinks the billing account"; on AWS, Budget Actions that detach permissions. Because this amounts to taking the service down, configure it only against an isolated experimental project or account. Wired into the same container as production, the kill switch becomes the outage.

Code for a Function That Can't Run Away

Cutting the self-trigger loop usually takes a few lines. Write to a different destination, and then state explicitly that the function does not react to its own output.

def on_upload(event):
    name = event["name"]
    if name.startswith("thumbs/"):      # never react to our own output
        return
    if event.get("metadata", {}).get("generated_by") == "thumbnailer":
        return                          # second guard: check the marker too
    make_thumbnail(name, dest=f"thumbs/{name}",
                   metadata={"generated_by": "thumbnailer"})

The point is that there are two conditions. The prefix check alone works today, but the infinite loop comes back the moment someone changes the destination path. The metadata marker survives that change. In a domain where a mistake turns into an order-of-magnitude bill overnight, the redundant guard is the cheap option.

Functions that call metered third-party APIs deserve a per-execution ceiling as well.

MAX_CALLS = 50                      # per-invocation cap on paid API calls

def handler(items):
    for item in items[:MAX_CALLS]:
        call_paid_api(item)
    if len(items) > MAX_CALLS:      # don't drop silently — record and escalate
        log.warning("truncated %d items", len(items) - MAX_CALLS)

When an unexpectedly huge input arrives, do not quietly process all of it. Truncating and logging leaves you something to recover from.

How This Shows Up on the Job

You need this material when you are an indie developer trying to ship inside a free tier, a backend engineer being asked "so what does this cost per month?" in a design review, and an SRE or platform engineer isolating the cause of a bill that spiked.

The settings you actually touch map across providers. On the function: timeout (never leave it at the long default), memory (decided by CPU-bound vs I/O-bound), maxInstances / reserved concurrency (your spend ceiling), minInstances, retry configuration and a dead-letter queue. On the project: budgets and alert thresholds, the automated path to disabling billing, and the cost breakdown report (Cost Explorer / billing reports). Label — or tag — your resources, and you can trace a spiking line item back to the feature that caused it.

The pitfalls that turn into incidents, roughly in order of how often they bite:

• minInstances > 0 breaks the premise. A single line added to hide cold starts deletes the biggest advantage you had: zero cost when nobody uses the thing. Add it only after pricing the standing charge.
• Logs become the top line item. Leave verbose debug logging on in production, meet real traffic, and log ingestion charges can exceed compute charges. Logs are not free.
• Database connections run out first. Serverless spins up as many independent environments as your concurrency allows, and if each opens a connection you hit the connection limit. A pool or a proxy is the standard answer; why connections are so expensive in the first place belongs to Database Internals — What Happens Behind a Single Line of SQL.
• Free tiers are usually shared per account. Splitting into multiple projects doesn't multiply your free tier — it separates blast radius and gives you a unit you can switch off.
• Never design to find out from the invoice. Watch invocation count and concurrency graphs rather than money. Those move by the minute, which is fast enough to catch an exponential on the way up.

The question that comes up in design reviews is: for this workload, serverless or always-on, and how do you decide? The reasoning is the one above — compare monthly invocations against the break-even $N^{*}$, then look at the shape of the traffic (peak-to-average ratio). If the peak is tens of times the average, fixed capacity has to be provisioned for the peak, so any comparison based on averages alone will be wrong.

Summary

• Serverless is fundamentally about scaling to zero: the deeper the troughs, the cheaper it is; a workload that runs hot around the clock is cheaper on fixed capacity
• What's counted is invocations, GB-seconds, and egress. Set memory by whether the function is CPU-bound or I/O-bound
• Failures come from the tail, not the average — self-triggering and retries grow exponentially, and are already out of hand by the time they're visible
• A budget alert is a smoke detector, not a sprinkler. Put concurrency caps, timeouts, and automated billing shutoff in place before you go live

Next in Cloud & Ops: seeing inside a running system — metrics, logs and traces. Having covered how to stop it, we move on to how to notice.