2026-08-13·★ MEMBER·PAPER·9 min read

Paper Walkthrough: Mental World Modeling — A World Model That Advances Minds, Not Just Physics

You can track a scene's physics perfectly and still get the next human action wrong. A walkthrough of Mental World Modeling (MWM), which carries beliefs, intentions, emotions and norms as state variables so that every action updates physics and mind together — plus Mentis, the training-free baseline, and its numbers across eight models, straight from the paper.

#agents #world-model #theory-of-mind #pomdp #llm

ModalityimageTaskagents

Mental World Modeling

Primary source — what this article is built on

undefined2026-07-29→undefined2026-08-13same month

Mental World ModelingHao Fei, Yiran Zhao · 2026-07-29 · v1arXiv:2607.27201Paper page·PDF

undefined

World models enable a predictive substrate for planning and action, yet existing formulations merely answer a physical question: what/where it is, and how will it evolve. Human behavior, however, is driven by hidden mental state (what a person believes, wants, intends, feels, and considers socially permissible), so a model that tracks the physical scene but not what each agent knows and believes about it predicts the wrong action for the right-looking scene. We formulate Mental World Modeling (MWM), a generic theoretical framework that makes mental variables core components of a world model rather than posthoc rationales: MWM aintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions jointly update both components. We instantiate the framework in MENTIS, a training-free and fully inspectable baseline that decomposes the process into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. On a manually constructed, quality-controlled dataset of situated decision scenarios spanning text, image, and sounding-video stories, experiments with 8 modern LLM-based world models demonstrate that explicitly modeling the mental state is essential for predicting human decisions. Deeper analyses further expose the bottlenecks of current mental world modeling. We expect MWM as a next stage of world modeling, from simulating physical scenes to simulating the minds that act in them.

Getting the scene right and the action wrong

A mug is sitting on the desk. Its owner looks away, and while they aren't watching, someone moves the mug to the shelf. Where do they reach next?

Physically, the mug is on the shelf. But the owner didn't see it move, so they still believe it's on the desk — and that's where their hand goes. A model that tracks object positions flawlessly, but doesn't track what's inside this person's head, will confidently predict a reach toward the shelf. The paper frames this as predicting the wrong action from a scene it got exactly right (§1, Figure 2).

Mental World Modeling (MWM), proposed here by Hao Fei and Yiran Zhao, is the framework meant to close that gap. The canonical world models are Dreamer-style latent-state models, Sora/Genie-style video generators, and Marble-style 3D spatial models — and the paper's reading is that all three aim at the same physical substrate of objects, positions, shapes and motion, leaving the humans inside the world treated as just another kind of moving object (§2.1). MWM's claim fits in one sentence: the next state of the world is mental as well as physical.

The intuition: minds are state variables, not after-the-fact commentary

Most existing Theory of Mind research — inferring what others believe or intend — asks "what does this person believe?" as a static, one-shot question. MWM asks for something different: beliefs, intentions, emotions and norms held as state variables that evolve over time, updated alongside the physical state every time somebody acts (§2.2).

Why insist on both? The paper's proof sketch (Property 3.1) runs like this. Two situations can share a physical state and still have different true action distributions if the beliefs differ — and a physics-only model, unable to tell them apart, has no choice but to return the same prediction. Run it the other way and a mind-only model can't distinguish situations that differ in what's visible or reachable. Hence the need for a joint physical–mental state (§3.1).

The formal skeleton: a POMDP with mental variables bolted on

MWM builds on the partially observable Markov decision process (POMDP: a decision-making framework in which only part of the state can be observed) (§3.3).

s_t = \big(s_t^{\mathrm{phy}},\, s_t^{\mathrm{ment}}\big)

(1)

Stated in words: the world at time $t$ is not one photograph but a two-column ledger — one column for what is where, one for who is thinking what. The mental column isn't an annotation on the physical one; it has equal standing, and it gets carried into the next moment intact.

This declares the world state $s_t$ to be a pair: a physical component (objects, people, spatial relations, environment) and a mental one (each individual's mind, the group's mind, social relations, the mood of the room). An individual mind decomposes further into typed fields — belief, attention, goal, intention, emotion, disposition, norm, action constraints, and so on (Eq. 8). A single step then flows through three functions.

s_t \xrightarrow{\ \Omega_\epsilon\ } o_t^{\epsilon} \xrightarrow{\ \Pi_\epsilon\ } a_t^{\epsilon} \xrightarrow{\ T_\theta\ } s_{t+1}

(2)

Or in words, taking the arrows left to right: cut out the slice of the world this person can actually perceive, let them choose their next move from that slice and nothing more, and let the move carry the world into its next shape. Only the third arrow is allowed to rewrite every column of the ledger.

Read it as: observation generation $\Omega_\epsilon$ renders the first-person partial observation $o_t^\epsilon$ belonging to the target person $\epsilon$ ; the policy $\Pi_\epsilon$ produces an action from that observation and nothing else; the transition $T_\theta$ predicts the next joint state. What the target conditions is the observation, not the world state — which is exactly what lets an action driven by the false belief of someone who missed the mug's move fall out naturally. The policy is a probability distribution over candidate actions, and the evaluation measures the decision margin between the top choice and the runner-up (§6.2). The figure gives a feel for how peaked such a distribution gets.

FIG 1How peaked the distribution becomes when candidate-action scores are turned into probabilities. A flat distribution means a thin decision margin — a small scoring error flips the choice, which is why MWM's evaluation tracks decision margins and tie rates

Observation filters, action is a duet, transition couples the two

Observation generation is information filtering. You can observe someone hunched over and speaking quietly; you cannot observe whether that's shame or relief. Mental observation is a bundle of self-observation and ToM inference, and it is deliberately viewpoint-dependent: watching the same person cry, someone who knows they were expecting an award infers "disappointment," while someone who knows they were just rescued infers "relief" — and neither is required to match the true state (Eq. 14).

Paper Walkthrough: Mental World Modeling — A World Model That Advances Minds, Not Just Physics

Getting the scene right and the action wrong

The intuition: minds are state variables, not after-the-fact commentary

The formal skeleton: a POMDP with mental variables bolted on

Observation filters, action is a duet, transition couples the two

Members-only from here

References

Comments