Paper Walkthrough: Metis — A 'Memory Foundation Model' That Moves Agent Memory Inside the Model

Agent memory today is mostly bolted on from the outside via RAG. This paper proposes memory foundation models — models whose forward pass natively stores, forgets, and updates information — and builds Metis, the first prototype. A from-scratch walkthrough of how it works, how well it works, and where it breaks.

Metis: Memory Foundation Model

AI memory is still a sticky-note system

Imagine a brilliant colleague whose memory is wiped after every meeting. That is, roughly, today's LLM. The standard workaround is to keep transcripts of past interactions in an external store and paste the relevant bits back into the prompt — Retrieval-Augmented Generation (RAG). The model doesn't remember anything; it rereads a stack of sticky notes before every reply.

Human memory doesn't work like that. Experience soaks into the wiring itself, and remembering isn't a separate step from thinking. The paper "Metis: Memory Foundation Model" (MemTensor, Renmin University of China, National University of Singapore, and others; published July 2026) is an attempt to give LLMs that kind of soaked-in memory. The authors call the concept a memory foundation model and build the first prototype, Metis.

Three limits of bolt-on memory

The paper opens by listing three problems with external memory (§1).

1. Mismatched objectives: the memory module's job is "assemble a useful context," while the backbone's job is "continue the text it was given." The two are designed separately, so the memory may not surface what the model actually needs, and the model may not use what it gets optimally.
2. No end-to-end training: gradients cannot flow through discrete operations like "retrieve" and "concatenate," so you can't tune memory behavior directly from task performance.
3. Latency: retrieval, reranking, and stitching run on every request, adding inference delay.

Against this, the paper defines native memory by two components (§2):

• Native memory state: a persistent, evolving store that lives inside the model, represented as part of its parameters.
• Native memory procedure: remembering, forgetting, and updating carried out autonomously by the model's own forward computation — no external tooling.

The definition of "memory" here is strict: anything the model knew before the interaction started counts as knowledge, not memory. Only information acquired online, during the interaction, qualifies (§2.2). The paper's deeper framing is that memory is fundamentally a prediction problem — predicting how information received now will be used later (§1). And if it's a prediction problem, it should be learnable from data, like everything else.

The primitive underneath all the retrieval is one you already know from Transformers: the dot product. Worth refreshing the intuition before we open the machinery.

Inside Metis: two blocks share the work

Metis adds a Metis block to each Transformer layer, made of two parts (§3.2).

• Local memory block: the memory itself — a fixed-size matrix $\mathbf{M}_t \in \mathbb{R}^{d_k \times d_v}$ plus a normalization vector $\mathbf{S}_t$. These are the only dynamic parameters; they change from interaction step to step and start out as zeros.
• Hyper memory block: the writer. It holds a learned importance vector that scores which tokens are worth keeping, and projection matrices that turn selected content into memory keys, values, and queries. These are frozen after training and never change during interactions.

In other words: the rules for how to remember are baked in by training; only the contents of memory move at inference time.

Writing: select, compress, accumulate

When a step of the conversation finishes, Metis writes it to memory in three moves (§3.3).

First, the hidden states of each layer are scored by the importance vector, and a temperature-scaled softmax turns the scores into a distribution over "which tokens deserve to be remembered." Metis then keeps only the top tokens whose cumulative probability reaches a threshold $\rho$ — a compression step that distills a long input down to its essentials.

The selected tokens are projected into memory keys $\tilde{\mathbf{K}}_t$ and values $\tilde{\mathbf{V}}_t$ and folded into the memory matrix:

Written out in words: new memory = old memory, slightly faded, plus today's highlights. $\mathbf{M}_t$ is everything remembered up to now and $\mathbf{M}_{t+1}$ is what it becomes after this step; $\lambda$ (the discount factor) is the share of the old memory that survives, so the remaining $1-\lambda$ is exactly the room left for new material. $L'_t$ is how many tokens made the cut — dividing by it averages the new content, so a long-winded step can't shout over a terse one — and $\tilde{\mathbf{K}}_t^{\top}\tilde{\mathbf{V}}_t$ packs a key-to-value lookup table into a single matrix, with $\sqrt{d_k}$ keeping the magnitudes in range. In practice, the authors found that replacing this linear update with a Gated Delta Network (GDN)-based update (GDU) works better, and Metis ships with GDU (§3.3).

The crucial property: this update is pure forward computation. No gradients, no weight updates. At inference time every learned weight stays frozen, and only the memory state transforms as the model runs (Abstract, §3.3).

Reading: memory attention

Reading happens through memory attention (§3.3). The current input produces a dedicated memory query $\tilde{\mathbf{Q}}_t$, which is applied directly to the memory matrix:

Spelled out in words: hold the question up against the memory's lookup table, pull out whatever matches, and rescale so the answers stay comparable. $\tilde{\mathbf{Q}}_t$ is the question the model puts to its own memory, $\mathbf{M}_t$ is the accumulated lookup table, and multiplying them gives the raw answer; the $\mathrm{diag}(\tilde{\mathbf{Q}}_t\mathbf{S}_t)^{-1}$ factor then divides each row by how much stored material that query matched in total, so a query that overlaps many memories at once doesn't come back with an inflated readout. Because equation (1) stored key–value products, values whose keys resemble the current query are read out most strongly. This readout is then blended with ordinary self-attention using a mixing weight $\gamma$, so every token is produced by two eyes at once — one on the current context, one on the past (§3.3, Eq. 5).

The paper also shows that this memory attention approximates what you would get by virtually prepending summary tokens from past steps to the prompt (§3.4), and derives that the errors introduced by cramming many steps into a fixed-size state are all proportional to the similarity between the query and irrelevant steps (§3.5). That analysis is exactly why the dedicated, trainable memory query matters: it learns to push down similarity with the wrong memories. On efficiency, the ordinary attention, the memory read, and the memory write have no sequential dependency and can run in parallel, and since the state is fixed-size, read cost doesn't grow with the number of past interactions (§3.6).

What it ate to learn how to remember

Architecture alone doesn't produce memory behavior. The authors synthesized training data from 27 public benchmarks covering four operations — remember, forget, update, and reflect (composing multiple stored facts) — yielding 357,137 samples and about 406 million tokens after filtering (§4.1). On top of that, 609,443 auxiliary samples target two realistic failure modes: interference between similar facts, and "memory pollution," where stored values leak into answers that don't need them (§4.2).

Training is mid-training: the backbone (Qwen3.5 family) stays frozen while only the memory-related parameters learn. Three objectives shape the behavior — a reconstruction objective (store and recover content), an operation objective (execute the four operations correctly), and a regularization objective (suppress interference and pollution) (§5, §6.1).

How well does it work?

The main battleground is the no-context setting: past information never appears in the prompt, and the model must answer from its memory state alone (§6.2).

• Vanilla Qwen3.5 collapses without context — on the conversational benchmark LoCoMo (Gold), even the 27B model averages 0.07. The answers simply aren't in its weights.
• Metis-27B recovers to 26.74 in the same no-context setting, beating Temp-LoRA (a method that trains a temporary LoRA during inference; 4.24–11.72 depending on size) and $\delta$-Mem (10.79).
• On memory-operation benchmarks the gap widens: 73.77 on the Metis test set — approaching the 78.87 that Qwen3.5-27B scores with the full context pasted in.

The paper is equally frank that full context, when you can afford it, still wins (65.03 on LoCoMo (Gold)). The ablations are telling: removing the adaptive write-selection (which tokens get remembered) costs about 61% of overall performance — by far the biggest hit — meaning memory quality is mostly decided at write time, not read time (§6.3). The authors also report meaningful transfer to out-of-distribution benchmarks never used in data construction (§6.4).

The paper is honest about its weaknesses

A refreshing amount of space goes to failure analysis.

• Capacity walls: accuracy drops sharply as a single write gets longer, and degrades again as writes accumulate over a trajectory — the price of compressing into a fixed-size state (§6.6).
• Side effects on general ability: with an empty memory, Metis matches its backbone almost exactly. But after irrelevant information has been stored, general-task scores fall — strict instruction following (IFEval) drops by 22.18 points (§6.7).
• Inconsistent responses: after a "forget" instruction, the memory state is correctly cleared, yet the model's immediate reply sometimes still repeats the old fact (§6.9).
• The authors' own conclusion: native memory is not yet a replacement for external memory; hybrids combining both are the way forward (§8).

How this gets used in practice

If you build agents or run LLM infrastructure, this paper feeds directly into the "where should memory live?" design decision.

• To try it: code and checkpoints are public (GitHub: MemTensor/Metis; Hugging Face: IAAR-Shanghai/metis), built on Qwen3.5 4B/9B/27B. The knobs you'd touch are the discount factor $\lambda$ (how fast old memory fades), the blend weight $\gamma$ (ordinary vs. memory attention), and the write-selection threshold $\rho$ and temperature $\tau$.
• Efficiency angle: memory updates are a single gradient-free forward pass, so unlike inference-time training approaches such as Temp-LoRA, there's no backprop load on your GPUs. For storage, the memory matrix can be SVD-compressed to rank 64 (of 1024) with 99.9% performance recovery (§6.8) — making low-rank, per-user persisted memory states a realistic design.
• Pitfalls that become incidents: (1) Memory will degrade over long horizons, so never make the memory state the only home for information that must stay exact — contract terms, amounts, medical facts belong in a RAG store with the originals. (2) Accumulated irrelevant memory hurts instruction following (§6.7), so define explicit reset boundaries for memory state in your session design. (3) Because the reply right after a forget can still voice the old fact (§6.9), don't build your "deletion confirmed" UI on the model's own introspection.
• The interview question this prepares you for: "RAG versus parametric memory — what's the trade-off?" Freshness and auditability favor RAG; latency and end-to-end optimizability favor native memory. Answering along the lines of this paper's §1 and §8 holds up well.

Related articles

• For how the dominant bolt-on approach works, see RAG Fundamentals.
• For the short-term inference cache that often gets confused with memory, see The KV Cache from Scratch.
• For why agents need memory at all, start with LLM Agents from Scratch.