#scaling-laws
3 articles
01
·Inference & Serving·★ MEMBER·PAPER·15 min read
Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?
A study of looped Transformers that finally controls the comparison: per-token FLOPs, total parameters, and KV cache are all held close. The resulting recipe, SMELT, loops the middle half twice and reports 6.8–18.0% training-FLOPs savings on the compute-optimal frontier.
02
·Large Language Models·★ MEMBER·PAPER·10 min read
Scaling Skepticism — A Genealogy of the "Just Make It Bigger" Critique
A fair accounting of the case against "more parameters, more data" — data exhaustion, the reasoning wall, and the world-model dispute. The evidence for scaling gets the same scrutiny as the evidence against it, and every claim comes with the observation that would settle it.
03
·Large Language Models·★ MEMBER·PAPER·9 min read
Scaling Laws from Scratch — Why Making Models Bigger Makes Them Smarter (and When It Doesn't)
Scaling laws turn 'bigger is smarter' into an equation. This article walks from the Kaplan-era recipe to the Chinchilla paper (Hoffmann et al., 2022) — 400+ training runs, three independent estimation methods, and the 70B-vs-280B showdown — using only the paper itself as evidence.