#nf4

1 articles

01 ·Training & Alignment·FREE·PAPER·19 min read QLoRA — Fine-Tuning a 65B Model on One GPU A 65-billion-parameter model in 16-bit weights is 130 GB before you have trained anything. QLoRA gets that onto a single 48 GB card by storing the frozen weights in a 4-bit format shaped like the weights themselves, quantizing the quantization constants, and paging the optimizer to CPU — while gradients still flow through those 4-bit weights into a small 16-bit adapter.