#int4

1 articles

01 ·Inference & Serving·FREE·8 min read LLM Quantization from Scratch — Why Losing Precision Doesn't Break It Cut a 16-bit weight down to 4 bits and the model keeps writing. Why doesn't it break? Starting from two numbers — scale and zero point — through the reason weights survive but activations resist (outliers), the fork between PTQ and QAT, what INT8 and INT4 actually do, and the single most common evaluation accident: signing off on perplexity alone.