Paper Walkthrough: Code World Model — Putting a Coding Agent in Charge of the World

A ground-up reading of Code World Model, which hands the *evolution* of a world to executable code and its *appearance* to a video model — including the proxy interface that joins them, and the limits the authors admit to.

Code World Model: Coding Agent as World Brain

The paper here is "Code World Model: Coding Agent as World Brain" (Yiwen Chen, Guosheng Lin, Chi Zhang, Westlake AGI Lab; arXiv:2608.25927, published 26 August 2026).

Its argument, in brief. World models are supposed to simulate how an environment changes under actions and events, but video-based world models learn dynamics from visual observations — and observations reveal outcomes, not the knowledge, rules and mechanisms that produced them. That makes it hard to keep consequences alive over time or to support coherent, open-ended evolution. So the authors split the job in two. A coding agent acts as the world brain: it reasons about events and their consequences, and writes executable code that maintains persistent world state and advances it in a rule-consistent way. To connect that executable state to image synthesis, they introduce a proxy representation that encodes frame-wise spatiotemporal constraints and compiles into a proxy video, which conditions a video model to render high-fidelity observations. They also build data pipelines that produce aligned proxy–observation pairs from gameplay and from real-world footage. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications coming from simple interactive worlds the coding agent built, while keeping rich visual detail and dynamics.

Video only ever shows you the outcome

Why take such a detour at all? The paper's own example makes it concrete (§1).

In a fantasy world, a player assassinates a city's ruler and walks away. That single event should keep shaping succession, local order, trade, faction relations, and the beliefs, goals and behaviour of many non-player characters. Those consequences continue to unfold while the city is off-screen, tangle with later events, and may only become visible when the player returns much later.

Now consider a video world model — one that generates the next frames conditioned on actions or prompts. The paper's objection (§1): video records the observable outcomes of world dynamics, while the world knowledge, commonsense rules, inter-entity relations, long-horizon plans and consequence-propagation mechanisms that produced them are not directly observable. Gameplay footage is the sharpest case. Every frame is rendered by executing fixed game code, yet that executable logic is discarded after rendering and can be seen only indirectly, through its sparse visual traces. Many important state changes also happen off-camera, or over spans longer than the model's temporal context. Scaling the data gives you more observed outcomes, not more evidence about mechanism.

There is a timescale mismatch on top of that: the paper notes that the training context windows of current video models are typically shorter than one minute, while many causal processes that require reasoning unfold over days or years of world time (§3.1).

The metaphor: a director who writes, a crew who draws

Code World Model is a division of labour.

• The coding agent is the director. It decides what happened and what should therefore change in the world, and writes that decision down as executable code.
• Code is the rulebook on set. Positions, numeric attributes, collisions, cooldowns — the dense, repetitive, deterministic updates run without calling the director back.
• The video model is the art department. It turns the settled blocking into pictures, with materials, lighting and fine motion.

In the paper's phrasing: code determines what occurs in the world and which outcomes affect subsequent interaction; the video model generates how those outcomes unfold and appear (§1). The split pays off because the two capabilities were already grown on different data — language models get world knowledge and long-horizon reasoning from text and code, video models get priors over appearance, motion and interaction from large-scale footage.

The formulation: splitting state into executable and visual

Follow the equations (§3.1, Formulation). Let $S_t$ be the complete world state at time $t$, $A_t$ an action applied to the world by a player, the environment or another agent, and $O_t$ the visual observation a user or downstream agent receives. The conventional world model reads:

Which says, in words: the current state and action give you the next state, and the next state gives you the next picture.

Code World Model cuts $S_t$ in two:

$S_t^{\mathrm{exe}}$ is the executable state — the evolving world program, entity attributes, rules, relations, event history, and every other variable that can be operated on directly. $S_t^{\mathrm{vis}}$ is the visual state: the appearance, motion and other visual information produced by the video model that has to stay consistent over time.

The two are updated by different but coupled processes. Writing $\mathcal{T}_{\mathrm{AC}}$ for the joint agent–code transition and $G_\theta$ for the video model:

Plainly: the substance of the world is advanced deterministically by code, and the picture is then sampled from a generative model conditioned on that result. The paper adds a caveat worth keeping: writing $S_{t+1}^{\mathrm{exe}}$ as an input to $G_\theta$ expresses an abstract conditional dependency only — it does not mean raw code or the full executable state is fed to the network (§3.1). How the relevant information is selected and represented is the subject of the next section.

Thin agent, thick code

The point the paper keeps returning to is that it separated two kinds of work that run at different frequencies (§1, §3.1):

• Sparse but semantically complex reasoning — interpreting new events, connecting them to world knowledge, character relations and social structure, deciding which entities and mechanisms are affected, planning consequences over long spans. That is the coding agent's job.
• Dense, repetitive execution — maintaining positions, attributes, schedules, cooldowns, collisions and numeric rules, and turning high-level decisions into concrete state updates. That is code's job.

Why bother separating them? As a world grows, thousands of state variables need frequent updates — characters, objects, and global processes like weather, traffic and scheduled events. Invoking a language model for each one would be too expensive to sustain real-time responsiveness (§3.1). And some outcomes have to be rule-consistent and reproducible. The paper's example lands the point: a video model can render the appearance of an attack, but whether it hits should be decided from world state — the distance to the target and the attack range (§1). Without stable rules, players cannot form reliable expectations about how their actions affect the world.