#architecture

24 articles

01 ·Paper Deep-Dives·★ MEMBER·PAPER·7 min read Paper Deep Dive: From Evidence to Action: How Tool-Using Agents Fail — Reading the Failure Diagnostics and Interventions SafeActBench splits tool-using agent failures into a chain of investigate, act, and hand off. This deep dive reads the paper through its appendices: how each failure diagnostic is defined, what agents do when decisive evidence is removed, and whether structuring evidence actually helps. 02 ·Paper Deep-Dives·★ MEMBER·PAPER·7 min read Paper Explained: EVISKILL — Rewriting an Agent's Playbook from Replayable Evidence A framework for automatically revising an LLM agent's skill document that pins every proposed edit to the exact trajectory steps that motivated it, then re-runs only those steps to check the edit actually works. It posts the best accuracy in 14 of 18 benchmark–model settings. 03 ·Paper Deep-Dives·★ MEMBER·PAPER·6 min read Paper Explained: Persona Dosing — Asking an LLM for a Personality Trait by the Score A steering coefficient says nothing about how strongly a trait will show up. PersonaDose trains one description-conditioned controller, then measures a strength-to-score map after training, so you can ask for "sycophancy at 60" instead of guessing a number. 04 ·Paper Deep-Dives·★ MEMBER·PAPER·10 min read Paper Explained: False Frontiers — Diagnosing and Breaking Co-Cheating in Self-Evolving Search Agents When the model that writes questions and the model that answers them start agreeing on the same wrong answers, internal reward climbs while real accuracy doesn't. This paper measures that 'co-cheating' with an external audit and breaks it with CrossFit: let a solver that never trained on a source grade the questions from it. Mechanism, results, cost, and limits. 05 ·Paper Deep-Dives·★ MEMBER·PAPER·10 min read Paper Explained: RayOrch — Batching Children Across Parents on GPUs Without Losing Track of Who Owns What Split PDFs into pages and videos into clips, batch them across sources on GPUs, and still route every result back to the right parent in the right order. A from-scratch walkthrough of RayOrch: its vocabulary, state machine, commit rule, and measured results. 06 ·How Transformers Work·★ MEMBER·PAPER·9 min read Paper Deep Dive: Your Transformer Can Hold Two Thoughts at Once — Blend Two Texts and the LLM Still Remembers Both Average the embeddings of two unrelated texts, run a single forward pass, and an LLM's output still keeps both streams' next-token candidates near the top. We walk through the paper's evidence that this 'superposition linearity' fades during pretraining, comes back with light fine-tuning, and can be partially decoded into two continuations. 07 ·How Transformers Work·★ MEMBER·PAPER·7 min read Paper Explained: LimiX-2 — A Tabular Foundation Model That Learns the Whole Table, Not Just the Answer Column LimiX-2 from the ground up: the shift from PFNs that predict a single target to Contextual Mechanism Networks that learn a table's joint distribution, cell-level representations, pretraining on synthetic causal data, and what the three benchmarks do and don't show. 08 ·How Transformers Work·★ MEMBER·PAPER·15 min read Paper Walkthrough: SAS — Training a Sparse-Attention Block Selector End-to-End With the Language Modeling Loss The selector that decides which context blocks a query reads has traditionally been trained by imitating the dense model's attention distribution. SAS injects the selector's scores into the softmax as log-space gates, so the gradient that Top-K used to block flows straight from next-token prediction into the selector. We follow the paper through its four-axis ablation, the gradient formulas behind each choice, and the 10.6–15.5 point GPQA-Diamond gain at a 1024-token budget. 09 ·Paper Deep-Dives·★ MEMBER·PAPER·14 min read Paper Walkthrough: CosmoH2G — Transferring a Human Hand's Flips and Rotations to a Two-Finger Gripper Turning a video of a human hand into a two-finger gripper motion has mostly worked for flat, simple tasks like pushing and placing. CosmoH2G (arXiv:2609.07498) collects 6,189 paired hand–gripper episodes focused on rotations and flips, then fixes the start and end poses before generating the motion in between. A walkthrough grounded only in the paper's text. 10 ·Large Language Models·FREE·PAPER·12 min read Mamba and State Space Models — Handling Sequences Without Attention Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention. 11 ·Large Language Models·FREE·PAPER·12 min read Mamba and State Space Models — Handling Sequences Without Attention Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention. 12 ·How Transformers Work·★ MEMBER·PAPER·11 min read Paper Explained: Why Gated DeltaNet Survives 4-Bit Quantization — NVFP4 W4A4 in a Hybrid 27B A walkthrough of the paper that tested — and overturned — the belief that the recurrent half of a hybrid LLM is too fragile for 4-bit quantization, and explained mechanistically why log-space gates and the delta rule erase quantization noise. 13 ·How Transformers Work·★ MEMBER·PAPER·11 min read Paper Walkthrough: Stop Anchoring to Frame One — Scal3R's Multi-Reference Relative Pose Query Long videos make online 3D reconstruction collapse. This paper traces the cause to extrapolation against a fixed first-frame anchor and fixes it by adding roughly 1% of trainable tokens to a frozen backbone. Explained from zero. 14 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper Walkthrough — RoboTok: Mining the Web for Demonstrations That Move Like Yours A ground-up walkthrough of RoboTok (arXiv:2609.03199), a data engine that pulls manipulation-relevant human demonstrations out of web video. Covers how DTW defines 'similar motion', how that ranking is distilled into an embedding for inner-product search, and what the results do and don't show. 15 ·How Transformers Work·★ MEMBER·PAPER·16 min read Qwen3.8-Next design report explained (arXiv:2608.30320) A ground-up read of the Qwen3.8-Flash-Next report: 125B total parameters with 6B activated, the GDN hybrid, Qwen Sparse Attention (7.6x faster prefill at 1M context), the Gated Residual and n-gram embeddings — judged on loss, cost and training stability. 16 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Walkthrough — J-Zero: Growing the Challenger, the Solver, and the Judge Together from Zero Data A model that writes its own problems, solves them, and grades them — with no external data and no human labels. J-Zero's twist is that the grader learns too, which is what lets it keep improving past the two-iteration wall where prior methods stall. 17 ·Engineering Process·★ MEMBER·9 min read Architecture Decisions — Telling Apart What You Can Undo From What You Cannot How to separate the design decisions you can cheaply reverse from the ones you can never take back. Measuring reversibility, writing ADRs that record what you gave up, and a template for putting trade-offs into words — from first principles. 18 ·Paper Deep-Dives·★ MEMBER·PAPER·13 min read Paper Walkthrough: Turning Game Development into a Verifiable Trajectory Data Engine — RLHEV and AWoMo A paper arguing that what world models lack is not data or compute but cheap grading. We work through RLHEV — post-training on game-engine checks fused with developer accept/reject decisions — from first principles to equations, interactive figures, and the reported numbers. 19 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper walkthrough: Apodex 1.1 — scaling agents around completed work Not a bigger model and not more thinking time — Apodex 1.1 scales two other surfaces: the environments an agent learns in, and the way work is organised across agents. A walkthrough from the task contract to the AgentOS delivery gate, the numbers, and the limits. 20 ·How Transformers Work·FREE·PAPER·10 min read The Transformer, End to End — One Token's Journey from Embedding to Output Between typing a prompt and getting a word back, a single token gets handed a vector, rewritten in room after room, and finally turned back into language. Tokenizing, embedding, position, attention, feed-forward, residuals, and the output head — walked as one continuous trip rather than a pile of parts. 21 ·Paper Deep-Dives·★ MEMBER·PAPER·13 min read Paper Deep Dive — Large Discovery Models: giving an LLM a value signal for what to try next An LLM can propose candidates but cannot reliably grade its own proposals. Large Discovery Models tilts the LLM's proposal distribution by an acquisition value drawn from a Gaussian process surrogate, so search keeps working when every evaluation is expensive. From the equations to the reported numbers. 22 ·Paper Deep-Dives·★ MEMBER·PAPER·11 min read Paper Walkthrough: WorldClaw — Agents That Build Walkable, Editable 3D Open Worlds from a Single Sentence Type 'a tropical pirate stronghold' and get a large-scale 3D world you can walk through — with every building still an editable, standalone asset. A walkthrough of Tencent Hunyuan's WorldClaw, from its plan → terrain → regional-objects pipeline to the render-and-inspect loops where agents check their own work. 23 ·Paper Deep-Dives·FREE·PAPER·9 min read Mixture of Experts (MoE) from Scratch — Routing and Load Balancing in the Switch Transformer A ground-up explanation of Mixture of Experts, the sparse architecture behind today's largest LLMs, built strictly from the Switch Transformer paper: the router math, expert capacity, the load-balancing loss, and the three tricks that make sparse training stable. 24 ·Paper Deep-Dives·★ MEMBER·PAPER·8 min read Paper Deep Dive: AskChem — Changing the Unit of Search from Papers to Provenance-Carrying Claims An NYU-led systems paper that swaps the retrieval unit of chemistry literature search from papers to claims grounded by a DOI and a verbatim quote. We walk through the 2.4M-claim index, evidence graph, and hybrid search design, the evaluation that took DOI resolution from 88.3% to 100%, and the paper's honest limitations.