#sparsity

1 articles

01 ·Distillation & Compression·★ MEMBER·PAPER·10 min read Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller Model There are three roads to a smaller model: coarsen the number grid (quantization), remove weights outright (pruning), or rebuild the thing at a smaller size (distillation). A head-to-head comparison on compression ratio, accuracy, and implementation cost — and why, when you stack them, training-based methods go first and quantization goes last.