Learn › Media & Compression
Coding Theory
Lossless vs lossy, entropy coding, Huffman, arithmetic coding
01
·Coding Theory·★ MEMBER·9 min read
Entropy Coding from Scratch — From Huffman to Arithmetic Coding
"Information content becomes code length" — that one line is all of compression. From Shannon's bound to building a Huffman code by hand, the integer-bit ceiling, the arithmetic coders and ANS that break it, and what is actually running inside JPEG, PNG, H.264 and zstd. Ending with how to diagnose a compression ratio that has stopped improving.
02
·Coding Theory·FREE·11 min read
Error Correction from Scratch — Sending Data on the Assumption It Will Break
Starting from a single parity bit, this article builds up Hamming distance, syndrome decoding and Reed–Solomon codes with no assumed background, then explains why a smudged QR code still scans — and where ECC bites you in production.
03
·Coding Theory·★ MEMBER·11 min read
LDPC and Turbo Codes — The Error Correction Behind 5G and Deep Space
Shannon proved in 1948 that these codes exist, and nobody could build one for 45 years. Starting from zero background, this article unpacks the three ideas — soft decisions, log-likelihood ratios, and belief propagation — that finally reached the theoretical limit, and shows what actually runs in 5G NR and deep-space links.