#quantization
8 articles
01
·How Transformers Work·★ MEMBER·PAPER·10 min read
Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27B
A walkthrough of the paper that tested — and overturned — the belief that the recurrent half of a hybrid LLM is too fragile for 4-bit quantization, and explained mechanistically why log-space gates and the delta rule erase quantization noise.
02
·Distillation & Compression·★ MEMBER·PAPER·10 min read
Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller Model
There are three roads to a smaller model: coarsen the number grid (quantization), remove weights outright (pruning), or rebuild the thing at a smaller size (distillation). A head-to-head comparison on compression ratio, accuracy, and implementation cost — and why, when you stack them, training-based methods go first and quantization goes last.
03
·Inference & Serving·★ MEMBER·11 min read
Cutting Inference Cost in Practice — What to Do First
Quantization and distillation both work, but in most shops there are two or three levers left that cost you nothing in quality. Break the bill into four numbers, then reorder the work: cache, batch, shorten, compress. What each lever actually buys, and what breaks when you take them out of order.
04
·Inference & Serving·★ MEMBER·8 min read
LLM Quantization from Scratch — Why Losing Precision Doesn't Break It
Cut a 16-bit weight down to 4 bits and the model keeps writing. Why doesn't it break? Starting from two numbers — scale and zero point — through the reason weights survive but activations resist (outliers), the fork between PTQ and QAT, what INT8 and INT4 actually do, and the single most common evaluation accident: signing off on perplexity alone.
05
·Inference & Serving·★ MEMBER·8 min read
LLM Quantization from Scratch — Why Losing Precision Doesn't Break It
Cut a 16-bit weight down to 4 bits and the model keeps writing. Why doesn't it break? Starting from two numbers — scale and zero point — through the reason weights survive but activations resist (outliers), the fork between PTQ and QAT, what INT8 and INT4 actually do, and the single most common evaluation accident: signing off on perplexity alone.
06
·Accelerators·FREE·9 min read
How Numbers Are Represented — From FP32 to FP8 and INT4
Open up the sign, exponent and mantissa fields and one rule falls out: the exponent buys range, the mantissa buys precision. From there — why bfloat16 was invented, why FP8 ships in two flavours, and what integer quantization actually destroys. Ends with which format to pick for training versus inference, and how to notice degradation before your users do.
07
·Accelerators·FREE·9 min read
How Numbers Are Represented — From FP32 to FP8 and INT4
Open up the sign, exponent and mantissa fields and one rule falls out: the exponent buys range, the mantissa buys precision. From there — why bfloat16 was invented, why FP8 ships in two flavours, and what integer quantization actually destroys. Ends with which format to pick for training versus inference, and how to notice degradation before your users do.
08
·Image Codecs·★ MEMBER·9 min read
Why JPEG Degrades — The DCT and Quantization from Scratch
When a photo shrinks to a tenth of its size, what exactly was thrown away? Colour conversion, 8×8 blocks, the DCT, the quantization table and the zigzag scan — followed by what the number in "quality 90" really is, and where blocking and mosquito noise are born. Ending with the decisions people who ship images actually make.