Paper explained: BDH-CQ — an AI that thinks without words. Recurrent memory plus latent reasoning resets ARC's cost frontier

BDH-CQ (arXiv:2608.09888) never verbalizes its intermediate reasoning: demonstrations are written into a recurrent memory, and the answer is computed by iterating in a high-dimensional latent space. A 150M-parameter model reaches 29.5% pass@2 on ARC-AGI-1 at $0.0007 per task, breaking the reported cost–accuracy Pareto frontier. We explain the paper using only its own text as evidence.

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

Thinking out loud is expensive

Today's reasoning models (think ChatGPT's reasoning mode) solve hard problems by writing out their intermediate steps as text — chain-of-thought (CoT). That design carries a structural tax: every intermediate state must be projected through a discrete vocabulary, emitted one token at a time autoregressively, and read back in before computation can continue. The longer the reasoning, the more tokens, latency, and compute it burns (§1).

The paper we're reading today, BDH-CQ (arXiv:2608.09888, from the research team at Pathway), takes the opposite road. Demonstrations presented at inference time are written into a recurrent memory on the spot, and the answer is worked out by iterative computation in a high-dimensional latent space. Nothing in between is ever put into words. With this design, a model of just 150M parameters reaches 29.5% pass@2 on the visual-reasoning benchmark ARC-AGI-1 at a computed cost of $0.0007 per task — less than a tenth of a cent — breaking through the previously reported cost–accuracy Pareto frontier (§1, §5).

ARC: a test of learning on the spot

ARC (the Abstraction and Reasoning Corpus) shows you a handful of input–output pairs of colored grids and demands that you infer the hidden transformation rule and apply it exactly to a new input. It was designed to measure skill-acquisition efficiency: not whether a system possesses a skill, but how little experience it needs to acquire one (§2).

Every answer is verifiable cell by cell, and every failure can be inspected visually. Because each task has several test inputs, you can distinguish "got one lucky hit" from "applies the rule consistently." As experimental material for studying in-context learning, it is about as controlled as it gets (§2).

An analogy: mental-arithmetic champions don't narrate

A soroban (abacus) mental-arithmetic champion moves an imagined board directly in their head. They beat someone who narrates every step — "add seven, carry the one…" — because they never pay the cost of converting intermediate results into words. The trade-off: nobody can see inside their head, so when they slip, you can only guess where from the final answer. That strength (speed, cost) and weakness (opaque process) is exactly BDH-CQ's character.

Mechanism 1: writing demonstrations into memory without touching the weights (§3.2)

A task provides $K$ demonstrations $D=\{(x_t, y_t)\}$ and a query input $x^\star$. BDH-CQ processes the demonstrations one by one, updating a recurrent memory $S$.

In plain terms: "mix the $t$-th demonstration $D_t$ into the current memory $S_{t-1}$ to produce the next memory $S_t$." The function $U_\theta$ decides how the mixing happens, and its weights $\theta$ stay frozen throughout inference. No gradient step ever runs; all of the "learning" happens inside this state variable. Think of $S$ as a notebook: the model rewrites one page per worked example, and the equation, in words, says that each new page depends on exactly two things — the previous page, and the example just read.

The paper positions this recurrent contextual state as playing the same role as the context-dependent associations that attention builds — with one decisive difference: there is no key–value cache that grows with context length (§3.2). Where a Transformer's KV cache balloons in proportion to the context, $S$ has a fixed size. Linear attention, the paper notes, is the conceptually simplest special case of this scheme: $S_t = S_{t-1} + U_\theta(D_t)$.

Mechanism 2: iterative reasoning in latent space (§3.3)

Once the demonstrations and the query have been ingested, BDH-CQ iterates inside a structured latent workspace $H$.

Reading it out: (1) encode the query $x^\star$ together with the memory $S_K$ into an initial state $H_0$; (2) apply the same transformation $F_\theta$ for $R$ rounds, refining the state; (3) decode only the final state $H_R$ into the answer $\hat{y}$. The division of labor — $S$ changes as evidence arrives and supports in-context learning, while $H$ is the workbench for the current query — is the skeleton of the system, and the intermediate states $H_r$ are never decoded into language. In everyday terms, $S_K$ holds the model's notes on the demonstrations and $H$ is the scratch pad for this one question; the middle equation is the load-bearing one, which says the scratch pad is rewritten $R$ times while the notes stay fixed, and only its final state is ever turned into an answer.

The foundation is BDH (the Dragon Hatchling), a post-Transformer architecture the same team introduced previously, built on high-dimensional positive activations, low-rank communication, and a recurrent associative state — "brain-inspired but not brain-imitative," in their own words (§3.1). One caveat the paper itself states plainly: the dimensions, exact update rules, and training recipe remain proprietary (§3.3, §4.1).

Training data and the headline result (§4, §5)

The 150M-parameter model is trained on an ARC-style mixture: the ARC-AGI-1 training set, RE-ARC, ConceptARC, ARC-Heavy, and ARC-GEN100K, combined with privately curated data and augmentations (§4.2). No evaluation-task demonstration pairs or task identifiers participate in training, and no parameters are updated at inference (§1).

On the 400-task public ARC-AGI-1 evaluation set, the system scores 29.5% pass@2 (24.25% pass@1), taking roughly 0.85 H200 GPU-seconds per task — $0.00070 per task at $3 per H200-hour (§5). Against ARC Prize's July 2026 data, the paper reports this as about 57x cheaper than GPT 5.6 Luna (Low) (34.2% at $0.040 per task), and still about 11x cheaper after accounting for that model's 80% price cut (§5). It also cites HRM/TRM — transductive systems that run optimization on the evaluation tasks themselves — at $1.48/$1.76 per task (§8). An independent black-box audit by co-authors from Bielik and NYU reproduced the 29.5% score without access to the model weights (§5).

What it can do, and where it stumbles (§6)

What makes this paper worth reading is that it doesn't stop at a single score — it dissects, with controlled experiments, what the model actually learned from the demonstrations.

A concept-by-concept profile. On ConceptARC (16 concept families x 10 tasks), the system reaches 59.4% pass@2 at the task level but 77.9% at the test-pair level. Families like ExtendToBoundary hit 9/10, while Copy and Order sit at 2/10 (§6.1).

The boundary of extrapolation. Using controlled tasks generated after freezing the model, the authors turn one difficulty dial at a time. Line propagation and motif copying stay perfect at 48/48 even beyond the demonstrated range, but ordering collapses from length six onward (1/24 at length eight), and nesting drops to 29/36 at depth five (§6.2). The decisive experiment is an intervention: adding a single demonstration at the test's complexity restores depth-five nesting from 19/24 to 24/24, and lifts length-eight ordering from 0/24 to 13/24 (§6.2, Table 3). The collapse was mostly a failure to extrapolate beyond what the demonstrations covered — not a lack of execution capacity.