compression

Definition

Representing information — or a model — in fewer bits while preserving what matters. Practically it covers quantization, pruning, and distillation, all aimed at cutting memory and latency for deployment. Theoretically it runs deeper: good prediction and good compression are formally linked, so some papers evaluate language models by how well they compress held-out text rather than by task accuracy.

Explainers using this term

• Paper Explained — LatentPress: Feeding Compressed Context Straight to a Frozen LLM, Neither as Text Nor as PixelsLatentPress: Context Compression Beyond Text and Vision
• Compression Is Prediction Is Intelligence — LLMs Through Information TheoryLanguage Modeling Is Compression
• Neural Compression — The Codec That LearnsEnd-to-end Optimized Image Compression