Paper Explained — LatentPress: Feeding Compressed Context Straight to a Frozen LLM, Neither as Text Nor as Pixels

A method that writes long conversation histories and documents into continuous memory tokens a frozen LLM reads directly — no text summary, no rendered image. On LongMemEval it beats uncompressed evidence at 7.70x compression. Explained from first principles.

LatentPress: Context Compression Beyond Text and Vision

Who is compressed context supposed to be readable for?

The paper behind this article is "LatentPress: Context Compression Beyond Text and Vision" (Zhengze Zhou, Hejian Sang, arXiv:2609.01507, September 2026).

Here is what its abstract claims. Compressed context is usually carried as human-readable text, or as rendered images that must be decoded — even when the consumer is a language model. LatentPress introduces a third representation: it writes conversational histories and long documents into continuous memory tokens that a frozen decoder reads directly through its input-embedding interface, with no text reconstruction at inference. A small reader-matched writer compresses 4–16× while training only an adapter — 4.2M to 26.2M parameters, roughly 0.1% of the decoder. On LongMemEval it reaches 0.504 accuracy at 7.70× compression, against 0.490 for uncompressed evidence, beating text summaries (0.184) and OCR-based compression (0.426 down to 0.312). Writing costs 43 ms per conversation, about an order of magnitude faster than summarization or OCR reconstruction, and reading is 5–9× faster than raw context or cached OCR.

So the question is not "how high can the compression ratio go." It is: what should cross the boundary between stored context and the model that consumes it?

An analogy: when your successor is not a person

Long-running assistants and agents accumulate more history than they can afford to reread — instructions, dialogue, plans, tool calls, observations, environment feedback. A later decision usually depends on a small slice of it.

The standard approach resembles a job handover. You rewrite years of records into a prose summary for whoever comes next. Or you print the pages, photograph them, and let your successor OCR the images back into characters. Both assume the final step lands on something a human can read.

If the successor is a language model, that assumption is optional. What the model actually consumes is not characters but embedding vectors. Text is convenient for people and interoperable across systems, but a model does not require its stored or compressed context to be human-readable (§1). That is the paper's starting point.

Splitting the problem: Write and Read

The paper separates context use into two operations (§1):

• Write — map text into a compact state
• Read — supply that state to a frozen decoder for downstream QA

The abstraction covers both conversational histories and long documents. Crucially, the paper states that it does not attempt to replace retrieval, reflection, update policies, or conflict resolution in a complete memory system. It only asks what representation should cross the boundary. Miss that framing and the later experimental setup — where the correct evidence is handed to the model — will look like cheating rather than scoping.

Mechanism (1): entering through the embedding interface

Let a context $x=(x_1,\ldots,x_T)$ be a sequence of segments — dialogue turns or document chunks. A frozen decoder $f_\theta$ answers a question $q$ from a compact representation of that context.

Equation (1) in words: a writer — with trainable parameters $\phi$ and a per-segment compression rate $\pi$ — squeezes the context $x$ into a short sequence of continuous vectors $m$, and the decoder reads one concatenated sequence, the soft tokens followed by the embedded question, and writes the answer straight out. Because $m$ lives in the decoder's own embedding space, there is no stage that turns it back into text (§2.1).

This is exactly where it parts ways with lossy compression like JPEG. JPEG is useless until you reconstruct the image. Play with the figure below to feel how reconstruction degrades as you push the ratio — the reconstruction step itself is what LatentPress removes at inference.

Mechanism (2): where to spend the budget — role-based rates

The rule $\pi=(k_1,\ldots,k_T)$ decides how many neighbouring token positions get pooled into one soft token. The paper deliberately keeps this rule simple and hand-specified: the point is to test the direct-read interface, not to optimize the compression schedule (§2.2).

There are two rules. Uniform pooling sets $k_i=k$ for every segment. The role-based schedule instead exploits structure the input already exposes: for conversational memory it sets $k_{\text{user}}=1$ and $k_{\text{assistant}}\in\{8,16,32\}$. In other words, user turns bypass the writer and keep their raw token embeddings, while assistant turns get encoded and pooled away.

The conversation-level ratio is therefore not a preset budget — it emerges from the role and length mixture (§2.2, A.5).

Equation (2) in words: original history tokens $|\mathcal{C}|$ divided by the number of injected vectors, where $n_i$ is the length of turn $i$ and $k_{r_i}$ is the rate assigned to its role. Lossless user turns ($k=1$) spend one vector per token, so $\rho$ is driven by the assistant rate and the user/assistant token mix. The reported 4.62–7.70× figures are means over the evaluation set.

Mechanism (3): the loss that teaches what to keep

Only the writer head is trained. The architecture deep-copies the frozen decoder's bottom two transformer layers ($L=2$) to use as an encoder, then stacks a single $d\times d$ linear adapter on top. The adapter is identity-initialized, so the writer starts out close to raw token embeddings and departs from them only as training warrants (Appendix A.1). Trainable parameters: 12.849M for Qwen2.5-7B, 16.781M for Qwen3-8B, 4.196M for Qwen3-1.7B, 26.220M for Qwen2.5-14B. The borrowed layers and the whole decoder stay frozen.

For generic representation learning the objective is (§2.3):