memorize

Definition

To retain training examples verbatim instead of extracting generalizable structure. Large models demonstrably do this — sequences from the training corpus can be extracted back out of them — which makes it both a privacy problem, since personal data can be recovered, and an evaluation problem, since benchmark items absorbed during pre-training inflate scores. Memorization and generalization are the standard opposition, though where one ends and the other begins is itself an open question.

Explainers using this term

• Paper Walkthrough: Embodied-Navigator (TAMP-Nav) — Let the VLM Just Point, and Navigation Gets Both Faster and BetterEmbodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation