Paper Walkthrough: GigaBrain-0.7 — 37,000 Hours of Embodied Experience and a Three-System Robot Brain
A ground-up walkthrough of GigaBrain-0.7, which splits a robot foundation model into understanding, prediction/evaluation, and action, then pretrains it on 37,256.98 hours of embodied data in a single stage. Covers subgoal-image and progress-value conditioning, Soft Knowledge Insulation, and the real-robot numbers — strictly from the paper.
GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
Primary source — what this article is built on
undefined2026-08-16→undefined2026-08-27same month
GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System ArchitectureGigaBrain Team, Angen Ye, Axiang Sun et al. · 2026-08-16 · v1arXiv:2608.15875Paper page·PDFundefined
Vision-language-action (VLA) models have become a dominant paradigm for generalist embodied agents, demonstrating strong complex and long-horizon task completion in structured settings. Yet it remains an open question whether current VLA systems can benefit from more effective architectural design, scale to substantially larger and more heterogeneous data regimes, and achieve broader generalization across tasks and embodiments. To this end, we present GigaBrain-0.7, an embodied foundation model with substantially improved generalization across diverse robot embodiments. Specifically, GigaBrain-0.7 unifies understanding, prediction, and action through a three-system architecture, scales pretraining to over 37,000 hours of heterogeneous embodied data, and introduces one-stage alignment training that jointly optimizes vision-language understanding and multi-embodiment action generation. Compared with the preceding GigaBrain-0 series and prior state-of-the-art models including $π_{0.5}$, GigaBrain-0.7 achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates. In particular, on our in-house Maker H01 platform and mainstream robot embodiments, GigaBrain-0.7 demonstrates strong task adaptability and completion ability across both home and industrial scenarios. All training code and pretrained model weights will be released.
The robot that cannot fold a shirt
"Fold that shirt." A person starts within seconds; a robot finds this brutally hard, for three reasons. The shirt changes shape every time you grab it, so replaying a memorized trajectory gets you nowhere. The scene keeps returning to visually similar states, so a single frame cannot tell you which pass you are on. And when things go wrong, the robot has to notice that on its own and recover.
The dominant approach today is the VLA (Vision-Language-Action) model: take a large model that understands images and language, and have it emit actions (see our TurboVLA walkthrough). The paper's complaint starts one step past that. Current VLAs stay centered on reactive observation-to-action prediction, with little machinery for anticipating what happens next, and little for judging whether the current behavior is actually working (§1). GigaBrain-0.7 tries to fill that gap with three systems that each do a different job.
The metaphor: three cooks in one kitchen
Picture a busy kitchen run by three people.
- The planner (System 2) looks at what is on the counter and breaks the order down into "pick up the red hat."
- The taster (System 3) predicts "in a few seconds it should look like this," and scores whether the current attempt is getting closer to done.
- The hands (System 1) take that instruction plus the prediction and the score, and actually move the joints.
In the paper's terms these are understanding and planning, prediction and evaluation, and action and control. Concretely: System 2 is a PaliGemma2 (3B) VLM, System 1 is the same PaliGemma2 (3B) plus a dedicated Action Expert (0.5B), and System 3 is a 5B world value model built on GigaWorld-1 (§4).
What matters is that the three are trained under separate objectives and exchange information only through fixed, narrow interfaces. System 2 hands over a sentence describing what to do next; System 3 hands over one predicted future image and a single bit for "is progress going up." Nothing vague flows between them, which means you can tell which part is helping and which part broke.
"Bigger backbone, better robot" does not hold
So which VLM belongs underneath System 1? The paper compares PaliGemma2 (3.5B total), Qwen3.5 (5B), and Gemma 4 (8.5B) on real-robot tasks (§6.2, Tab. 4). The answer is not tidy. Gemma 4 scores best on desk cleaning and fruit picking — and 0% on shirt folding. PaliGemma2 is the only backbone with a nonzero shirt-folding rate (30%), which is why the paper builds on it.
The paper flags its own caveat: the configurations differ in both model size and image resolution, so this is not a controlled scaling study (note to Tab. 4). Still, you can read off where the authors chose to invest — not in a bigger backbone, but in data and in division of labor.
Data: forcing 37,256.98 hours into one shape
After cleaning, the embodied trajectory corpus totals 37,256.98 hours: real robots 20,535.65 h (55.12%), UMI hand-held human demonstrations 8,251.83 h (22.15%), EGO first-person human video 2,862.36 h (7.68%), simulation 1,453.92 h (3.90%), and world-model-generated data 4,153.22 h (11.15%). The real-robot slice alone spans 16 robot types, 1,810,101 episodes, and 2,077,837,071 frames, and runs alongside 271,976,674 image-text/VQA samples (§3.1).
Heterogeneous data interferes if you just pile it together, so the normalization steps are spelled out: convert everything to LeRobot v3.0, fix the dimension ordering as left arm → right arm → head → waist → base or legs, represent rotations in the continuous 6D format, and rewrite language instructions with GLM-5.1 (§3.2). Cleaning is four-stage: q01/q99 outlier removal (quantile normalization is used in training, so extreme values stretch the range and squash normal action signals), removal of long stationary segments, end-effector pose alignment using collected URDFs, and tracing abnormally high training losses back to the source video to blacklist the sample.
Inside System 1, and the trick of throttling the gradient
System 1 is a Mixture-of-Transformers that runs a vision-language (VL) stream and an Action Expert (AE) stream side by side at every layer (§4.2). The VL stream keeps causal self-attention so the pretrained VLM's autoregressive interface stays intact; the AE stream attends bidirectionally over the concatenation of both streams.
Comments
Sign in to comment