Paper Explained: SpatialBlock — Teaching Vision-Language Models Spatial Sense with 15,000 Block-Stacking Puzzles
A paper that improves spatial reasoning in vision-language models using nothing but 15,000 synthetic block-stacking questions — no real-scene 3D labels. We walk through how the data is built, the two training recipes, what actually helped, and where it falls short.
Key points
- SpatialBlock-15k is a set of 15,000 synthetic problems covering 3D-to-2D projection, viewpoint transformation, and structural combination.
- In the human study, 31 people solved 30 test questions with 95% average accuracy, while GPT-5 scores 43.8, Claude-Sonnet-4.5 46.3, and Gemini-2.5-flash 36.4.
- Training on real-scene spatial QA depends on per-object position and depth labels, often produced by external models, which makes it costly and noisy.
- Color changes are added as cues that push the model to pick one object as an anchor and read everything else relative to it.
- Models trained on it, whether answering directly or reasoning first, clearly beat their baselines and generalize to real-world spatial tasks.
SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem
Primary source — what this article is built on
undefined2026-09-07→undefined2026-09-271 mo later
SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking ProblemSoohyun Ryu, Sohee Kim, Eunho Yang · 2026-09-07 · v1arXiv:2609.07064Paper page·PDFundefined
Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks, yet their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images -- referred to as spatial intelligence -- remains limited. Existing approaches attempt to address this gap by using real-scene spatial question answering datasets that require dense geometric annotations. However, constructing such labels is costly, time-consuming, and often noisy due to reliance on external perception modules. In this work, we propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks. We introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset further incorporates controlled color modulation as visual cues to encourage anchor-based reasoning in visually complex conditions. Experiments demonstrate that LVLMs trained on our dataset through either direct answering or reasoning-based prediction significantly outperform baselines and generalize to real-world spatial tasks, despite the dataset's synthetic and compact nature. Code and data are available at https://github.com/rsoohyun/SpatialBlock.
Can toy blocks teach spatial sense?
The paper is "SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem" by Soohyun Ryu, Sohee Kim, and Eunho Yang (KAIST, September 2026).
Here is the abstract in brief. Large vision-language models (LVLMs) do well on many visual tasks, but they are still weak at spatial intelligence — mentally rebuilding the 3D structure of a scene from 2D images and reasoning about it. The usual fix is to train on real-scene spatial QA, which needs dense geometric annotations: expensive, slow, and noisy because they often come from external perception models. The authors instead borrow from human development and teach foundational spatial skills through block-manipulation puzzles. They build SpatialBlock-15k, 15,000 synthetic problems covering 3D-to-2D projection, viewpoint transformation, and structural combination, and add controlled color changes as cues that push the model to reason relative to an anchor. Models trained on it — whether answering directly or reasoning first — clearly beat their baselines and generalize to real-world spatial tasks.
Easy for people, hard for models
The starting point is a finding from developmental psychology: children build spatial cognition by playing with blocks (§1). Current models, however, fall apart on exactly these puzzles. In the paper's human study, 31 people solved 30 test questions with 95% average accuracy (§A.2). On the same test set (SB-Bench, 600 multiple-choice questions), GPT-5 scores 43.8, Claude-Sonnet-4.5 46.3, and Gemini-2.5-flash 36.4 (Table 1).
Existing remedies either bolt on a 3D module or train on real-scene spatial QA. Both depend on per-object position and depth labels, often produced by depth estimators and other external models — which, the paper argues, makes them costly and noisy (§2.2).
Three kinds of block puzzles
The paper splits the skill into three parts — reconstructing the whole structure including hidden supporting blocks, imagining the result of a transformation, and predicting what two parts look like when joined — and pairs each with a question type (§3.1).
| Question | What it asks |
|---|---|
| Q1: 3D-to-2D projection | Which 2D drawing shows this structure from a given side? |
| Q2: Viewpoint transformation | What does it look like after a rotation or flip? |
| Q3: Structural combination | What shape do you get when two structures are joined? |
Color as an anchor
With single-color blocks, the task is pure geometry. But in real scenes, the paper notes, people pick one object as an anchor and read everything else relative to it (§3.2). So color is added not as decoration but as a functional cue.
- Q1: blocks are colored by depth, so matching the silhouette isn't enough — you have to track front-to-back order through color
- Q2: one block gets a distinct color, and the question is where it ends up after the transformation
- Q3: the attachment point is shown by same-colored blocks overlapping across two separate images
Comments
Sign in to comment