Paper Walkthrough: Code as Worlds — An Agent That Writes the World Down as Runnable Code

A close read of Code-as-World, which represents physical worlds as executable code and searches for world hypotheses through a propose–execute–render–verify loop. The three parts of an EWR, the agentic discovery loop, the QuantiPhy results, and the limits the paper itself admits.

Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning

"What Happened" and "Why It Happens" Are Different Problems

The paper this article walks through is "Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning" (arXiv:2608.27549, published 27 August 2026 — a technical report from MirroS, Tsinghua University, Peking University and Nanyang Technological University).

Its claim, in summary: physical understanding and reasoning depend on being able to form compact, generalizable representations of the world. Today's vision-language models (VLMs) can recognize and explain a wide range of physical events, but they carry no explicit representation of the underlying machinery — object states, physical parameters, governing dynamics — so they cannot reliably reason about how the world will evolve or how it will respond to intervention. The paper therefore proposes Code-as-World, which represents a physical world as an executable world representation. Write physical composition, dynamic evolution and visual appearance as code, and you get an abstraction of the world that is compact, quantitatively grounded and controllable. To build such a representation from multimodal observations — a natural-language description, or a real video — the authors design an agentic discovery loop that proposes, executes, renders, verifies and revises. Finally, verified executable worlds are used as supervision to teach VLMs quantitative physical reasoning.

The framing in §1 is the clearest part: the phenomenon–mechanism dichotomy. A phenomenon states what happened; a mechanism explains why, and predicts what would happen under different conditions. A play-by-play commentator is a master of the former, which says nothing about their ability to draw the blueprint.

Every Existing World Representation Has a Hole in It

Section 2 lines up the three established families.

Pixels (video generation). If you only optimize future-pixel prediction, the model never has to decide whether a change on screen came from camera motion or object motion, or whether an object was briefly occluded or ceased to exist. A visually plausible future is not necessarily a physically correct future.

3D (reconstruction, inverse graphics). Forcing a model to recover geometry, viewpoint and appearance is a strong constraint, but reconstructability does not imply interpretability. Recovering an object's 3D shape does not explain why it falls once its support is removed.

Natural language. "A person picks up a cup" is an excellent abstraction — it keeps entities, actions and relations while discarding clutter. But precise geometry, trajectories, contact relations and physical parameters are hard to encode in a handful of discrete tokens. Language is a semantic interface, not a complete physical state.

The strengths are complementary, the paper argues. What is needed is a representation that is semantic like language, structured like reconstruction, and able to model temporal evolution like a generative model.

Writing the World as Runnable Code

Hence Code-as-World. Code is a structured, compositional abstraction that holds entities, relations, physical parameters and events explicitly as operations over states (§3). Concretely, an Executable World Representation (EWR) has three components (§3.1).

Plainly: a single world $p$ can be written as a triple — what is in it ($\mathcal{C}$), how it changes ($\mathcal{E}$), and how it looks ($\mathcal{A}$). Here $\mathcal{P}_{\mathrm{exec}}$ is the space of all valid executable worlds.

What the equation says, in words: do not hold a world as one picture or one video — hold it as three separate documents, its contents, its motion and its look. Only the three bundled together make an executable world, and $\in\mathcal{P}_{\mathrm{exec}}$ is the condition that the bundle is in a form a simulator can actually run.

• $\mathcal{C}$: objects, geometry, metric dimensions, mass, friction, gravity. Floors and walls are written as static physical entities too, so they participate in support, contact and collision.
• $\mathcal{E}$: initial states, temporal changes, key events, simulation duration. Executing it produces contacts, collisions, velocity changes and termination conditions.
• $\mathcal{A}$: camera, background, materials, lighting, frame rate, resolution, rendering configuration. This component does not alter the physical process — only how it is presented.

The split pays off because each component can be inspected, edited and executed independently. Section 4.3.2 shows the payoff directly: change only the initial velocity direction of a bowling ball and get a different trajectory; re-render the same collision from a global view or from either car.

Discovery Is a Loop, Not a One-Shot Guess

Recovering this representation from incomplete observations is fundamentally an inverse problem. The paper reaches for abduction: searching, from noisy and incomplete evidence, for the hypothesis that best explains it while obeying a principle of simplicity — the way scientific discovery has proceeded from heliocentrism to Newton's laws (§1, §4.2).

So world representation is formulated as an agentic discovery loop rather than a single prediction. Because code is executable, each representation can be treated as a testable world hypothesis. One iteration has five stages (§4.2, Algorithm 1):

1. Propose — update the EWR from the evidence $\eta$, the current hypothesis, and the previous round's feedback $\Delta$
2. Instantiate — compile it into simulator-ready parameters $\theta$
3. Execute — run it to get a full state trajectory $\tau$, with contacts, collisions and event outcomes explicitly recorded
4. Render — turn $\tau$ into predicted observations; for video input, also project into depth maps, masks and image-plane trajectories
5. Verify — compare prediction against evidence at selected key frames and aggregate the discrepancies into $\Delta$

$\Delta$ then tells the agent locally which component to revise next. The loop terminates when the current EWR explains the input sufficiently well and parsimoniously; if the iteration budget runs out first, the hypothesis is rejected.