#safety

12 articles

01 ·Paper Deep-Dives·★ MEMBER·PAPER·9 min read Paper Explained: Refusal in Language Models Is Mediated by a Single Direction — saying "no" lives on one line inside the model When chat models turn down harmful requests, that behavior is carried by a single direction in the residual stream. This walkthrough reads the paper as an interpretability result: how a difference of mean activations finds the direction, why removing it stops refusal, and why adding it makes models refuse harmless requests across 13 models. 02 ·Paper Deep-Dives·★ MEMBER·PAPER·9 min read Paper Explained: Refusal in Language Models Is Mediated by a Single Direction — saying "no" lives on one line inside the model When chat models turn down harmful requests, that behavior is carried by a single direction in the residual stream. This walkthrough reads the paper as an interpretability result: how a difference of mean activations finds the direction, why removing it stops refusal, and why adding it makes models refuse harmless requests across 13 models. 03 ·Paper Deep-Dives·★ MEMBER·PAPER·7 min read Paper Deep Dive: From Evidence to Action: How Tool-Using Agents Fail — Reading the Failure Diagnostics and Interventions SafeActBench splits tool-using agent failures into a chain of investigate, act, and hand off. This deep dive reads the paper through its appendices: how each failure diagnostic is defined, what agents do when decisive evidence is removed, and whether structuring evidence actually helps. 04 ·Agents·FREE·PAPER·6 min read SafeActBench explained (arXiv:2610.07753) — tool agents fail before execution A correct outcome does not mean the action was supported by evidence gathered beforehand. Across 656 SafeActBench cases and ten model-harness configurations, static action judgment scores 83.7-97.7 while interactive execution falls to 47.3-72.6 — and most failures start before execution. 05 ·Evaluation & Judging·★ MEMBER·PAPER·10 min read Reward Hacking — Whatever You Measure Is Where It Breaks The moment you pick a metric, that metric starts to rot. This piece explains why Goodhart's law is statistically unavoidable, walks through real failures from boat races that spin in circles to RLHF verbosity, sycophancy and hardcoded unit tests, and covers how to detect the gap between optimization pressure and true performance. 06 ·Paper Deep-Dives·FREE·13 min read When Proxies Stop Being Good Enough — Reading August 2026's Eight Autonomous Driving Papers Together A cross-cutting read of eight autonomous-driving arXiv papers from late August 2026. Three groups independently stop measuring safety in expectation, two add an observation channel outside the ego vehicle's own history, and two genuinely don't fit the story. 07 ·Large Language Models·FREE·PAPER·9 min read Why Language Models Hallucinate — The Mechanics and What Actually Helps Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation. 08 ·Large Language Models·FREE·PAPER·9 min read Why Language Models Hallucinate — The Mechanics and What Actually Helps Confident fabrication isn't a bug — it falls straight out of next-token prediction. This piece takes the mechanism apart layer by layer: no truth term in the loss, lossy compression that fails at the edges, sampling that draws from the tail, and scoring rules that pay models to guess. Then it ranks the countermeasures that actually work: grounding, mechanical citation checking, constrained output, and uncertainty estimation. 09 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps. 10 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps. 11 ·Security·★ MEMBER·PAPER·9 min read LLM Security — Prompt Injection and How to Actually Defend Against It A language model cannot tell your instructions apart from words printed on a document it was asked to read. This article works from that single fact to direct injection, indirect injection, the tool boundary that decides your blast radius, and the layered defence that actually holds. 12 ·Security·★ MEMBER·PAPER·9 min read LLM Security — Prompt Injection and How to Actually Defend Against It A language model cannot tell your instructions apart from words printed on a document it was asked to read. This article works from that single fact to direct injection, indirect injection, the tool boundary that decides your blast radius, and the layered defence that actually holds.