vram

Definition

The memory on a GPU board. Weights, activations, gradients, optimizer state, and the KV cache during generation all have to fit there, which makes VRAM the first hard limit on what you can train or serve — quantization, offloading, and gradient checkpointing all exist to work around it. It is a separate constraint from compute throughput: FLOPs determine how fast a job runs, VRAM determines whether it runs at all.

Explainers using this term

• Paper Walkthrough: TurboVLA — Kick the LLM Out of the Loop and Run a Robot Policy at 32 Hz on an RTX 4090 with Under 1 GB of VRAMTurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM