MP3 and Psychoacoustics — The Science of Sounds You Cannot Hear

MP3 does not throw away sounds that aren't there — it throws away sounds that are there and inaudible. This piece works through the three holes in human hearing (absolute threshold, simultaneous masking, temporal masking) down to the equations, then follows SMR into bit allocation, the encoder's two nested loops, and the bit reservoir.

Stars Are Out in the Daytime, Too

You can't see stars at noon, but they haven't gone anywhere. The sun is simply too bright for that light to reach your perception. The same thing happens in your ears, and it is called masking. What MP3 discards is not sound that is absent — it is sound that is present and inaudible.

The previous article, Audio Compression from Scratch, followed the skeleton: transform (MDCT) → quantize → entropy-code. This one pulls out the middle of that chain — the psychoacoustic model, which decides every frame how coarsely each frequency band may be rounded. Once you see it, the single line of config that says "128 kbps" turns into what it really is: the average of an enormous number of per-band decisions.

The Ear Is a Measuring Instrument Full of Holes

Human hearing is remarkably sensitive, but as an instrument it is riddled with defects. The useful part is that those defects are reproducible enough to measure. There are three holes.

1. Absolute threshold of hearing — sounds below a certain level are simply not registered, and that level differs by orders of magnitude across frequency
2. Simultaneous masking — while a loud sound is playing, quiet sounds near it in frequency disappear
3. Temporal masking — quiet sounds just after (and slightly before) a loud one disappear

Start with the first. In a silent room, raise the volume from zero and at some point the tone becomes audible. Plot that boundary against frequency and you get a U-shaped curve, lowest — that is, most sensitive — around 3–4 kHz, rising steeply on both sides. The dip sits there because the ear canal, roughly 2.5 cm long and closed at one end, resonates in that region. It is no accident that screams live in that band.

Terhardt's approximation is the standard way to write the curve.

Here $f$ is frequency in Hz and $T_q(f)$ is the sound pressure level at which that frequency becomes audible. Read in words: it is a sum of three parts — the first term is the wall on the bass side, the second carves the dip near 3.3 kHz, the third is the wall on the treble side. Anything below this curve can be deleted outright and nobody notices.

Critical Bands — The Ear's Own Frequency Ruler

Inside the cochlea, the basilar membrane is stiff and narrow at the entrance and floppier and wider deeper in. High tones shake the entrance, low tones the far end: position maps to frequency, so the membrane is a frequency analyzer.

Its resolution, though, is finite. A single point on the membrane responds over a range of frequencies, which makes the ear a bank of overlapping bandpass filters. The width of one filter is a critical band: roughly 100 Hz below 500 Hz, and about 20% of the center frequency above that. Twenty-four to twenty-five of them cover 20 Hz to 20 kHz. So "I can hear up to 20 kHz" is true, and also means you have about 25 distinguishable slots to work with.

Take one of those slots as a unit and you get the Bark scale, usually written with the Zwicker–Terhardt approximation.

$z(f)$ is the Bark value of frequency $f$, and a step of 1 in $z$ is exactly one critical band. Put in words, the formula swaps a physical ruler (hertz) for the ear's ruler (critical bands): the bass end gets stretched, the treble end compressed. On this axis, masking spreads out along near-straight lines, which is what makes the next section tractable.

When a codec says it "varies the coarseness of rounding per band," those bands follow critical bands. In MP3 they are called scalefactor bands — 21 of them for long blocks.

Images work the same way. A JPEG quantization table is essentially a psychoacoustic model for vision, baked into a fixed table instead of computed per frame. Sound doesn't play on a page, so see what coarser rounding does with your eyes first.