Paper Explained: Just-in-Time Memory — Curate Agent Memory When You Read It, Not When You Write It

A from-scratch walkthrough of JitMem, which stores raw agent trajectories and only summarizes them once the next task is known. Why that makes the curator easy to train, and exactly how far the reported numbers go.

Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents

What this paper claims

The paper is "Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents" by Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz, and Shafiq Joty, posted to arXiv in September 2026. The method is called JitMem.

Here is the abstract in plain terms. Most memory systems for agents curate at write time: when a task ends, its trajectory (the log of actions and observations) is boiled down into a fixed artifact — a reflection, a workflow, a skill, a reasoning strategy — which is later fetched by similarity search. That forces the system to decide what is worth keeping before it knows what the next question will be. Information gets thrown away irreversibly, and the summary is generic, since it has to serve every possible future task. Training such a write-time curator is also hard, because whether a storage decision was good only becomes clear when a relevant task shows up, possibly many tasks later — a long-horizon credit-assignment problem. The authors instead keep raw trajectories and defer curation to read time. Given the retrieved traces and the new task, a curator writes a compact, task-specific payload. Because that payload is used on the very same task, the curator can be trained directly from that task's success. On ALFWorld, WebShop, and τ²-bench, JitMem beats the strongest baseline by 16.2, 16.3, and 3.9 success-rate points respectively, and even an untrained curator is competitive with or better than the baselines.

An analogy: when do you write the travel notes?

Picture two frequent travelers. The first one, after each trip, writes a three-line "lessons learned" note and throws away the photos and receipts. Before the next trip, they reread those notes. That is write-time curation.

The trouble is that when they wrote the note, they had no idea where they would go next. If the Kyoto trip got reduced to "visit temples early," and the next trip is "Kyoto with a toddler, find somewhere to eat," the thing they actually need — where the station's nursing room was — is already gone.

The second traveler keeps everything. Once the next destination is decided, they go back through the old records and write a one-page brief for this trip. From the same Kyoto records, one trip gets a temple route and another gets restaurant tips. That is JitMem's read-time curation.

The intuition: only the timing changes

The two approaches use almost the same parts: a retriever, an LLM that summarizes, and an LLM that actually does the task (the paper calls it the executor). The only difference is whether summarization happens before or after the task is known.

Section 1 names two costs of write-time summarization. First, whatever you discard cannot be recovered later. Second, a single frozen summary has to be reused across many different future tasks. The paper's example: one household trajectory might teach one task a state-change pattern (heating or cooling an object) and teach another an object-placement strategy. A single record holds several possible lessons, and which one matters depends on a task you have not seen yet.

Let's get a feel for the retrieval step first. The widget below uses embedding search, whereas the paper itself uses BM25 (a classic word-overlap search) over task descriptions (§3). The authors say the framework places no constraint on the retriever, so treat this purely as a picture of "the top k past records get selected."

How it works: four stages

Section 3 considers a streaming setting where tasks arrive one at a time. Call the task at step $t$ $x_t$, the resulting record (alternating observations and actions) $\xi_t$, and the success reward $r_t \in [0,1]$. There are four components: a memory bank $\mathcal{M}_t$, a retriever $\mathcal{R}$, a curator $\pi_\phi$, and a frozen executor $\pi_L$. Only the curator is trained.