memory

Definition

Mechanisms for carrying information beyond the immediate input — external stores queried by retrieval, recurrent state, or summaries written back into the context. It matters because context windows are finite and expensive, so long-running agents and conversations need somewhere to put what does not fit. Watch the ambiguity: efficiency papers often mean GPU RAM, not a cognitive component.

Explainers using this term

• Paper Walkthrough — Beyond Retrieval: LatentStream Turns Retrieved Video Into Latent MemoryBeyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
• Paper Walkthrough: VoiceMem — A Left Brain and a Right Brain for Voice Agents, at Zero Added LatencyVoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction
• LLM Serving from Scratch — vLLM, Continuous Batching, and Not Letting the GPU IdleEfficient Memory Management for Large Language Model Serving with PagedAttention

In the textbook

• The Memory Hierarchy and the Memory Wall