cost

Definition

Compute, time, or money — and the first question is always which, and whether it is paid once or per request. Training cost is a one-off; inference cost recurs with every user, so at deployment scale the second usually dominates the argument even though the first makes headlines. A large share of systems research is not about new capability at all, but about holding quality fixed and driving this number down.

Explainers using this term

• Surviving GPU Out-of-Memory — Every Cause, Every FixTraining Deep Nets with Sublinear Memory Cost