Students Professional
JA EN

#compression

27 articles

01 ·Distillation & Compression·★ MEMBER·PAPER·10 min read Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller Model There are three roads to a smaller model: coarsen the number grid (quantization), remove weights outright (pruning), or rebuild the thing at a smaller size (distillation). A head-to-head comparison on compression ratio, accuracy, and implementation cost — and why, when you stack them, training-based methods go first and quantization goes last. 02 ·Coding Theory·★ MEMBER·11 min read LDPC and Turbo Codes — The Error Correction Behind 5G and Deep Space Shannon proved in 1948 that these codes exist, and nobody could build one for 45 years. Starting from zero background, this article unpacks the three ideas — soft decisions, log-likelihood ratios, and belief propagation — that finally reached the theoretical limit, and shows what actually runs in 5G NR and deep-space links. 03 ·Information Theory·★ MEMBER·PAPER·11 min read Compression Is Prediction Is Intelligence — LLMs Through Information Theory Training a model to guess the next token is training it to shrink a file. Route the prediction through arithmetic coding and "probability model" and "code" turn out to be the same object — which makes cross-entropy loss literally the size of the compressed output. Built up from zero, ending at why the Hutter Prize treats compression ratio as an intelligence test. 04 ·Media in Production·FREE·10 min read HLS and DASH — How Video Actually Reaches You Streamed video does not flow — a player downloads small files a few seconds long, one after another. What segments and manifests really are, the arithmetic behind how a player picks a rendition, where latency comes from, how low-latency modes cheat it, and why none of it works without a CDN. 05 ·Audio Codecs·★ MEMBER·PAPER·11 min read Why Opus Won — The Design of a Modern Audio Codec Audio codecs used to split cleanly into speech codecs and music codecs. Opus ended the split by stacking two engines, SILK and CELT, inside a single bitstream. This piece works up from zero: how linear prediction and the MDCT divide the spectrum between them, why CELT refuses to let band energy collapse, what actually sets a codec's latency, and the technical and institutional reasons Opus became WebRTC's mandatory codec. 06 ·Audio Codecs·★ MEMBER·PAPER·11 min read Why Opus Won — The Design of a Modern Audio Codec Audio codecs used to split cleanly into speech codecs and music codecs. Opus ended the split by stacking two engines, SILK and CELT, inside a single bitstream. This piece works up from zero: how linear prediction and the MDCT divide the spectrum between them, why CELT refuses to let band energy collapse, what actually sets a codec's latency, and the technical and institutional reasons Opus became WebRTC's mandatory codec. 07 ·Video Codecs·★ MEMBER·PAPER·11 min read Neural Compression — The Codec That Learns JPEG's quantization table and H.264's prediction modes were tables people worked out by hand. Neural compression hands that design job to the data instead. The two pillars — an autoencoder and an entropy model — the trick that gets around non-differentiable rounding, the hyperprior, and what standards bodies are actually arguing about on the road to JPEG AI. 08 ·Audio Codecs·★ MEMBER·PAPER·12 min read Neural Audio Codecs — EnCodec and the Foundation Under Speech LLMs MP3 and Opus were hand-designed around what the ear cannot hear. Neural audio codecs learn the coding itself and turn sound into a finite alphabet of integers. From vector quantization to residual VQ, EnCodec's bitrate arithmetic, and why VALL-E and AudioLM are built on top of it — starting from zero. 09 ·Image Codecs·★ MEMBER·10 min read WebP, AVIF, JPEG XL — The Image Codec Changing of the Guard Three successors clearly beat JPEG on efficiency, and the web is still full of JPEGs. This piece traces where each newcomer came from, why still-image codecs keep getting born out of video codecs, and the structural reason a lingua franca doesn't die just because something faster shows up — down to how you actually compare codecs and ship the files. 10 ·Image Codecs·★ MEMBER·10 min read WebP, AVIF, JPEG XL — The Image Codec Changing of the Guard Three successors clearly beat JPEG on efficiency, and the web is still full of JPEGs. This piece traces where each newcomer came from, why still-image codecs keep getting born out of video codecs, and the structural reason a lingua franca doesn't die just because something faster shows up — down to how you actually compare codecs and ship the files. 11 ·Video Codecs·★ MEMBER·10 min read Motion Compensation from Scratch — Where 90% of Video Compression Happens What decides where a video codec's bits go isn't the transform or the quantizer — it's the prediction at the front door. What block matching actually minimizes, why motion vectors are not object motion, and how P/B reference structures and GOP boundaries turn into real streaming failures. 12 ·Video Codecs·★ MEMBER·10 min read Motion Compensation from Scratch — Where 90% of Video Compression Happens What decides where a video codec's bits go isn't the transform or the quantizer — it's the prediction at the front door. What block matching actually minimizes, why motion vectors are not object motion, and how P/B reference structures and GOP boundaries turn into real streaming failures. 13 ·Audio Codecs·★ MEMBER·10 min read MP3 and Psychoacoustics — The Science of Sounds You Cannot Hear MP3 does not throw away sounds that aren't there — it throws away sounds that are there and inaudible. This piece works through the three holes in human hearing (absolute threshold, simultaneous masking, temporal masking) down to the equations, then follows SMR into bit allocation, the encoder's two nested loops, and the bit reservoir. 14 ·Audio Codecs·★ MEMBER·10 min read MP3 and Psychoacoustics — The Science of Sounds You Cannot Hear MP3 does not throw away sounds that aren't there — it throws away sounds that are there and inaudible. This piece works through the three holes in human hearing (absolute threshold, simultaneous masking, temporal masking) down to the equations, then follows SMR into bit allocation, the encoder's two nested loops, and the bit reservoir. 15 ·Coding Theory·FREE·11 min read Error Correction from Scratch — Sending Data on the Assumption It Will Break Starting from a single parity bit, this article builds up Hamming distance, syndrome decoding and Reed–Solomon codes with no assumed background, then explains why a smudged QR code still scans — and where ECC bites you in production. 16 ·Video Codecs·★ MEMBER·7 min read From H.264 to AV1 — What a Codec Generation Change Really Involves H.264 has ruled video for two decades; AV1 arrived promising royalty-free efficiency. We unpack the transition along three axes — coding efficiency, compute cost, and licensing — and end with a practical framework for deciding which codec to ship, starting from zero background. 17 ·Video Codecs·★ MEMBER·7 min read From H.264 to AV1 — What a Codec Generation Change Really Involves H.264 has ruled video for two decades; AV1 arrived promising royalty-free efficiency. We unpack the transition along three axes — coding efficiency, compute cost, and licensing — and end with a practical framework for deciding which codec to ship, starting from zero background. 18 ·Audio Codecs·★ MEMBER·8 min read Audio Compression from Scratch — The Art of Discarding What You Cannot Hear How does MP3 sound 'the same' at less than a tenth of the size? We build up audio codecs from zero — auditory masking, the MDCT, and bit allocation — and finish with when to reach for MP3, AAC, or Opus. 19 ·Audio Codecs·★ MEMBER·8 min read Audio Compression from Scratch — The Art of Discarding What You Cannot Hear How does MP3 sound 'the same' at less than a tenth of the size? We build up audio codecs from zero — auditory masking, the MDCT, and bit allocation — and finish with when to reach for MP3, AAC, or Opus. 20 ·Video Codecs·★ MEMBER·8 min read Video Compression from Scratch — Motion Compensation and the GOP Compressing thirty separate photographs a second gets nowhere near today's delivery bitrates. Why frame differencing breaks, what a motion vector actually carries, I/P/B frames and the GOP, and what CBR, VBR and CRF each hold constant — ending at the keyframe interval a delivery engineer has to choose. 21 ·Video Codecs·★ MEMBER·8 min read Video Compression from Scratch — Motion Compensation and the GOP Compressing thirty separate photographs a second gets nowhere near today's delivery bitrates. Why frame differencing breaks, what a motion vector actually carries, I/P/B frames and the GOP, and what CBR, VBR and CRF each hold constant — ending at the keyframe interval a delivery engineer has to choose. 22 ·Image Codecs·FREE·8 min read Why PNG Does Not Degrade — Prediction Filters and Deflate What it means to shrink a file without changing a single bit. Starting from why no compressor can shrink everything, then PNG's five row filters as prediction-and-residual, Deflate (LZ77 plus Huffman), and why photographs resist it while screenshots collapse. Ending with a decision procedure for choosing between PNG and JPEG. 23 ·Image Codecs·FREE·8 min read Why PNG Does Not Degrade — Prediction Filters and Deflate What it means to shrink a file without changing a single bit. Starting from why no compressor can shrink everything, then PNG's five row filters as prediction-and-residual, Deflate (LZ77 plus Huffman), and why photographs resist it while screenshots collapse. Ending with a decision procedure for choosing between PNG and JPEG. 24 ·Image Codecs·★ MEMBER·9 min read Why JPEG Degrades — The DCT and Quantization from Scratch When a photo shrinks to a tenth of its size, what exactly was thrown away? Colour conversion, 8×8 blocks, the DCT, the quantization table and the zigzag scan — followed by what the number in "quality 90" really is, and where blocking and mosquito noise are born. Ending with the decisions people who ship images actually make. 25 ·Image Codecs·★ MEMBER·9 min read Why JPEG Degrades — The DCT and Quantization from Scratch When a photo shrinks to a tenth of its size, what exactly was thrown away? Colour conversion, 8×8 blocks, the DCT, the quantization table and the zigzag scan — followed by what the number in "quality 90" really is, and where blocking and mosquito noise are born. Ending with the decisions people who ship images actually make. 26 ·Coding Theory·★ MEMBER·9 min read Entropy Coding from Scratch — From Huffman to Arithmetic Coding "Information content becomes code length" — that one line is all of compression. From Shannon's bound to building a Huffman code by hand, the integer-bit ceiling, the arithmetic coders and ANS that break it, and what is actually running inside JPEG, PNG, H.264 and zstd. Ending with how to diagnose a compression ratio that has stopped improving. 27 ·Coding Theory·★ MEMBER·9 min read Entropy Coding from Scratch — From Huffman to Arithmetic Coding "Information content becomes code length" — that one line is all of compression. From Shannon's bound to building a Huffman code by hand, the integer-bit ceiling, the arithmetic coders and ANS that break it, and what is actually running inside JPEG, PNG, H.264 and zstd. Ending with how to diagnose a compression ratio that has stopped improving.