Paper Walkthrough: Metis — A 'Memory Foundation Model' That Moves Agent Memory Inside the Model

Agent memory today is mostly bolted on from the outside via RAG. This paper proposes memory foundation models — models whose forward pass natively stores, forgets, and updates information — and builds Metis, the first prototype. A from-scratch walkthrough of how it works, how well it works, and where it breaks.

Metis: Memory Foundation Model

AI memory is still a sticky-note system

Imagine a brilliant colleague whose memory is wiped after every meeting. That is, roughly, today's LLM. The standard workaround is to keep transcripts of past interactions in an external store and paste the relevant bits back into the prompt — Retrieval-Augmented Generation (RAG). The model doesn't remember anything; it rereads a stack of sticky notes before every reply.

Human memory doesn't work like that. Experience soaks into the wiring itself, and remembering isn't a separate step from thinking. The paper "Metis: Memory Foundation Model" (MemTensor, Renmin University of China, National University of Singapore, and others; published July 2026) is an attempt to give LLMs that kind of soaked-in memory. The authors call the concept a memory foundation model and build the first prototype, Metis.

Three limits of bolt-on memory

The paper opens by listing three problems with external memory (§1).

1. Mismatched objectives: the memory module's job is "assemble a useful context," while the backbone's job is "continue the text it was given." The two are designed separately, so the memory may not surface what the model actually needs, and the model may not use what it gets optimally.
2. No end-to-end training: gradients cannot flow through discrete operations like "retrieve" and "concatenate," so you can't tune memory behavior directly from task performance.
3. Latency: retrieval, reranking, and stitching run on every request, adding inference delay.

Against this, the paper defines native memory by two components (§2):

• Native memory state: a persistent, evolving store that lives inside the model, represented as part of its parameters.
• Native memory procedure: remembering, forgetting, and updating carried out autonomously by the model's own forward computation — no external tooling.

The definition of "memory" here is strict: anything the model knew before the interaction started counts as knowledge, not memory. Only information acquired online, during the interaction, qualifies (§2.2). The paper's deeper framing is that memory is fundamentally a prediction problem — predicting how information received now will be used later (§1). And if it's a prediction problem, it should be learnable from data, like everything else.

The primitive underneath all the retrieval is one you already know from Transformers: the dot product. Worth refreshing the intuition before we open the machinery.

Inside Metis: two blocks share the work

Metis adds a Metis block to each Transformer layer, made of two parts (§3.2).

• Local memory block: the memory itself — a fixed-size matrix $\mathbf{M}_t \in \mathbb{R}^{d_k \times d_v}$ plus a normalization vector $\mathbf{S}_t$. These are the only dynamic parameters; they change from interaction step to step and start out as zeros.
• Hyper memory block: the writer. It holds a learned importance vector that scores which tokens are worth keeping, and projection matrices that turn selected content into memory keys, values, and queries. These are frozen after training and never change during interactions.

In other words: the rules for how to remember are baked in by training; only the contents of memory move at inference time.

Writing: select, compress, accumulate

When a step of the conversation finishes, Metis writes it to memory in three moves (§3.3).

First, the hidden states of each layer are scored by the importance vector, and a temperature-scaled softmax turns the scores into a distribution over "which tokens deserve to be remembered." Metis then keeps only the top tokens whose cumulative probability reaches a threshold $\rho$ — a compression step that distills a long input down to its essentials.