Students Professional
JA EN

#linear-attention

2 articles

01 ·How Transformers Work·★ MEMBER·PAPER·10 min read Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27B A walkthrough of the paper that tested — and overturned — the belief that the recurrent half of a hybrid LLM is too fragile for 4-bit quantization, and explained mechanistically why log-space gates and the delta rule erase quantization noise. 02 ·How Transformers Work·★ MEMBER·PAPER·16 min read Paper Walkthrough: Designing Qwen3.8-Next — Accuracy, Efficiency and Stability as One Problem A ground-up read of the Qwen3.8-Flash-Next design report: the GDN hybrid, Qwen Sparse Attention, the Gated Residual and n-gram embeddings, judged the way the paper judges them — on loss, on cost, and on stability.