#gradient-checkpointing

1 articles

01 ·Inference & Serving·FREE·PAPER·11 min read Surviving GPU Out-of-Memory — Every Cause, Every Fix `CUDA out of memory` reports only the allocation that happened to fail last, which is almost never the culprit. We count what actually occupies VRAM in five buckets, derive the sixteen-bytes-per-parameter fixed cost of training, and work through the fixes in order of least damage: gradient checkpointing, optimizer compression, offloading, KV cache limits, and fragmentation.