Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27B

A walkthrough of the paper that tested — and overturned — the belief that the recurrent half of a hybrid LLM is too fragile for 4-bit quantization, and explained mechanistically why log-space gates and the delta rule erase quantization noise.

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

Questioning the intuition that recurrence is fragile

The paper read here is "Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM" (Sergii Kozyrev, Davyd Maiboroda / arXiv:2609.04098, 3 Sep 2026).

Here is its abstract in plain terms. Frontier open models are increasingly hybrid: most layers replace softmax attention with a linear-attention operator whose state is a fixed-size matrix summarizing the whole context — in Qwen3.8-27B, 48 of 64 layers are Gated DeltaNet (GDN) and only 16 are full attention. Yet every early 4-bit build of that model left the GDN block, and especially its decay and write-strength gate projections, at 8 or 16 bits, on the reasoning that errors inside a recurrence accumulate. The paper tests that reasoning by building Minima: NVFP4 W4A4 across all 496 linear layers, GDN included, and evaluating it on perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K. Minima matches BF16 within seed noise (5-task average $-0.52$) while being the smallest (17.5 GiB) and fastest-prefill (+14–19%) recipe compared — and its 32K perplexity gap shrinks with position. A four-part mechanism study explains why.

An analogy: broken telephone, or an overwritten whiteboard?

The fear about recurrence is the game of broken telephone. Each person mishears a little; a hundred people later nothing survives. Inject 4-bit rounding error into a layer that updates a state $S_t$ every step, the argument goes, and thirty thousand tokens later it must be wrecked.

What the paper shows is that GDN's state behaves more like a shared whiteboard. Each new token erases and rewrites the slot corresponding to its own key. Old writes do not merely fade — they get deleted. Errors disappear at the same rate.

Background 1: what rounding to 4 bits actually does

NVFP4 stores values in E2M1 (1 sign, 2 exponent, 1 mantissa bit) with one E4M3 scale per 16-element block (set to blockmax/6) plus one FP32 scale per tensor. W4A4 means both weights and activations are quantized at that granularity, so the GEMM runs on native 4-bit tensor cores (§2).

Two consequences matter later. First, a block's largest value fixes its scale, so an outlier degrades only its own 15 neighbors. Second, within a block $\max/\mathrm{RMS}\le\sqrt{16}=4$, which bounds how "one-hot" a block can get. The fastest way to feel what coarse quantization does is to watch it in image compression.

Background 2: what Gated DeltaNet computes

A GDN layer projects the residual stream through four linear maps: in_proj_qkv (which passes a depthwise causal convolution and SiLU before splitting into $q_t,k_t,v_t$), the output gate in_proj_z, and two scalar-per-head gate projections in_proj_a and in_proj_b. The gates are parameterized in log space (§2).

Read it aloud: $a_t$ is the raw projection output; pushed through softplus and an exponential it becomes $\alpha_t$, how much to keep (closer to 1 means forgetting less); and $b_t$ through a sigmoid $\sigma$ becomes $\beta_t$, how hard to write. Note where quantization noise lands — not on $\alpha$ itself but on $a_t$, upstream of the nonlinearity. That detail carries the whole result.

The per-head state $S_t$ (a $K\times V = 128\times128$ matrix) then evolves over $\ell_2$-normalized keys and queries:

The term in parentheses, $v_t-S_{t-1}^{\top}k_t$, is the point: it is the difference between what the state currently answers for key $k_t$ and the value $v_t$ we actually want stored. GDN does not accumulate $v_t$; it replaces whatever sat along the direction of $k_t$ with $v_t$. That is why it is called the delta rule.

So five weight matrices per GDN layer are candidates for quantization, and the community consensus protects $a$ and $b$ entirely while keeping the rest at 8 bits. Minima instead quantizes 496 tensors — 240 GDN, 64 attention, 192 MLP projections — leaving only embeddings, lm_head, the GDN conv1d, the norms, and $A_{\log}$/dt_bias in BF16, calibrated on a frozen set of 128 samples × 32K tokens and served text-only on a single RTX PRO 6000 with native NVFP4 support (§3).

Main result: quantize all of it, nothing breaks

Four checkpoints, one regime (vLLM 0.27.1, TP=1, a single RTX PRO 6000, FP8 KV cache, identical harness) (§4): BF16, Minima, and the two public builds Unsloth Dynamic v3 and RadixArk — both of which keep GDN and attention at FP8 W8A8 with $a$/$b$ in BF16 and quantize only the MLPs to NVFP4. Three things come out of the comparison, and the third one is where the paper's argument actually starts.