Paper Walkthrough: ComBodied Agents — Moving an Agent's Target from Software and Matter to the Person

Digital agents rewrite software state; embodied agents rewrite physical state. So who models the person? Combodied Agents bind perception, longitudinal memory, a Personal World Model, and an intervention policy into one closed loop, and move the success criterion from task completion to whether the user's agency survived. A position paper with no experiments — here is its skeleton, and its limits.

ComBodied Agents: a New Paradigm of Human-Centric Agentic AI

Someone skipped a dose, and nobody knows why

The scene the paper opens with (Abstract, §1) is almost aggressively mundane. An older adult misses a medication dose. A software agent can send another reminder. A robot can carry the pills over. Both are perfectly capable of acting. Neither, however, explains why the dose was missed. Did the person forget, get confused, experience side effects, or deliberately refuse? A repeat reminder is useless for someone who refused, and can be actively harmful for someone who is confused.

The paper reads this as a structural gap. Digital Agents are organized around transformations of software state; Embodied Agents around transformations of physical state. Neither makes a person's evolving state and agency the primary object of modeling, intervention, and evaluation (§1). Combodied Agents is the name given to that empty seat — a 38-page position paper from 22 co-authors whose aim is to outline a research programme, not to ship a model or a result.

Sorting agents by what they act on

The organizing axis is neither the interface nor the model architecture. It is the action substrate — the class of target states that primarily organizes a system's modeling, decision making, intervention, and evaluation (§2.2).

• Digital Agents: the target is digital state. GUI navigation, code editing, API calls
• Embodied Agents: the target is physical state. Robotic control, autonomous vehicles
• Combodied Agents: the target is the evolving human state and human agency

The subtle part is that the action substrate is explicitly not the same thing as the interface. A personal agent may drive a browser to schedule care and read a sensor to detect fatigue, yet neither the webpage nor the sensor reading is the point. Hence the paper's rule: agents "should therefore be classified by the target user state and success criteria that guide their behavior, rather than by their interface, underlying model, or mode of action" (§2.2). The paradigm is also distinguished from Human Digital Twins: rather than an exhaustive replica of a person, it keeps only the aspects a support context requires, as representations that are purpose-bounded, uncertainty-aware, and user-correctable (§1).

Five rungs from observation to intervention

§3 surveys eight modalities (language, speech, vision, physiology and biochemistry, motion, social and relational data, environmental context, institutional records), but the chapter's conclusion is not about modalities. It is about five layers the architecture must keep apart (§3.10).

observation → event → inferred state → predicted trajectory → authorized intervention

Using the paper's own example: an elevated heart rate is an observation. Elevated heart rate with slow recovery after exercise is a reconstructed event. Possible fatigue is an inferred state. Further delayed recovery if exercise continues is a predicted trajectory. Recommending that the person stop is a decision made by the Intervention Policy. Skip a rung and an observation hardens into a verdict, and a verdict into an action.

Back at the opening scene, the point lands. Forgot / confused / side effects / refused all stand on the same evidence. What is required is not picking one, but carrying all of them, unresolved, until the evidence or the person settles it.

The closed loop, in symbols

The true human state $H_t$ is unobservable. All the agent ever holds is $Z_t$, an uncertainty-bearing posterior built from the history of event-evidence records. And the transition §2.4 stresses hardest is this one.

$H_t$ is the person's current state, $H_{t+1}$ the next one, $a_t^{\mathrm{agent}}$ the agent's action, $a_t^{\mathrm{user}}$ the person's own action, $\Xi_t$ exogenous influences, and $T_H$ the process by which the next state falls out probabilistically.

Put in words: who the person is a moment from now follows from four things at once — who they are right now, what the agent did, what they did themselves, and what the world did around them. The two sides are joined by "is distributed as" rather than an equals sign because the same four inputs need not land on the same next state.

One line of consequence: the person's next state is not determined by the agent's action alone. Hit an API and the database row changes; people do not work that way. The formalization deliberately avoids assuming that an intervention has a deterministic or immediately observable effect (§2.4). This is where the paradigm departs from ordinary agent design.

Seven drawers of memory, ten kinds of action

Longitudinal Memory is not a chat log (Table 2). Episodic memory, semantic person memory, trajectory memory, goal and commitment memory, and relationship memory, plus intervention-response memory (was the advice accepted, refused, beneficial, harmful?) and user-control memory (do-not-remember rules, corrections, deletion requests). The last two are what existing memory features lack. The action space (Table 3) has ten entries — Inform / Remind / Recommend / Coach / Nudge / Reflect / Coordinate / Protect / Escalate / Execute — and the paper immediately qualifies it: the policy should not always intervene. Staying silent, asking for confirmation, or handing off to an authorized human can all be the right move (§2.4).

Personal World Model — trajectories, conditioned on choices

A Personal World Model (PWM) is defined as a purpose-bounded, individual-specific event-dynamics model (§4.1). Given a governed event history, current context, and a candidate scenario (user decisions, agent interventions, environmental change), it returns a calibrated distribution over the future.

$Z$ is the latent human state, $E$ future observable personal events, $Y$ scenario-relevant outcomes (adherence, goal progress, capability, safety, relationship quality, and agency preservation), $\mathbf{s}$ the candidate scenario, and $\Delta$ the prediction horizon.

Stated in words, the promise is this: name one scenario, hand the model the evidence gathered so far and the current context, and it returns how likely each way the coming stretch could unfold is — for the person's hidden state, for the events you would actually observe, and for the outcomes anyone cares about. It is not machinery for guessing the one true future.

The useful scenario sets are non-intervention, clarification, alternative intensities of support, and user acceptance or refusal (§4.2). What you compare is not "what happens next" but how the distribution over this person's trajectory shifts across explicit alternatives. And a PWM authorizes nothing: the policy picks nondominated actions only inside an admissible set $\mathcal{A}_t^{\mathrm{adm}}$.

$\mathcal{A}_t^{\mathrm{adm}}$ enforces consent, safety, reversibility, and escalation requirements, and always contains non-intervention, clarification, and referral. $\mathbf{U}$ is a vector rather than a scalar, precisely so trade-offs among benefit, capability, autonomy, and relationships are not collapsed.

Eq. 3 is a compact instruction which says: list only the actions the person has actually sanctioned, score each of them on several axes at once, and keep whichever ones nothing else beats on every axis. The ordering carries the argument — narrow the candidates first, optimize second. The paper is blunt about the consequence: safety and consent cannot be exchanged for higher engagement or average predicted utility (§4.2). A causal variant using $\operatorname{do}(\cdot)$ is also written down, immediately followed by the caveat that observed user behavior is confounded by motivation and hidden context — writing $\operatorname{do}(\cdot)$ "does not remove those confounders."

Personal data is sparse, biased, and goes stale

§4.3 handles the learning-side reality. Because personal data is sparse, training from scratch is inappropriate for most users. A practical PWM should start from population-level priors, adapt a limited set of personal components, detect drift, and support correction or reset.

The paper names the danger directly: local adaptation may overfit short-term behavior, reinforce incorrect inferences, and produce catastrophic forgetting (§5.2.3). Hence every update in local evolution must be inspectable, reversible, and safety-bounded (§5.4). Deployment follows the same logic, aiming at a Stage III architecture where the authoritative copies of memory, PWM, and policy live on the user's devices — though its defining property is not that every computation is local, but that personal interpretation and intervention authority stay under user-side control (§5.2.3).

Measuring whether agency survived

§6 makes a plain argument. Task completion and physical safety are necessary, but once an agent can alter a person over time, they are not sufficient. §6.3 lists eight agency-preservation metrics — autonomy preservation, contestability and correction, informed decision-making, capability preservation and growth, over-reliance and dependence risk, reversibility and accountability, boundary and consent respect, and relationship and social-world preservation. The measurements are concrete: do corrections propagate into future behavior; can the agent explain which memory or inference drove an action; independent post-support performance, retention tests, reduced scaffolding over time, transfer to new situations. And in high-stakes settings, strong average performance must not compensate for severe failures in autonomy, consent, dependency, or reversibility.

The proposed unit of evaluation is the scenario (§6.2), because the same intervention flips sign with context: a proactive reminder may support medication adherence, undermine autonomy under workplace monitoring, and deepen dependency in emotional companionship. CombodiedBench is likewise framed as a standardized protocol rather than a dataset, in which privacy leakage, unauthorized high-impact action, manipulation, harmful dependency, and irreversible action without consent are reported as non-compensatory critical failures (§6.4).

A single global memory is unsafe

§7 cuts the design space along three axes: which human state is targeted, which social relationship the person is situated in, and which role the agent takes within it. The heaviest implementation consequence is §7.3.1 — a single global memory is unsafe. Family history, intimate disclosures, health vulnerabilities, and workplace context require scoped memories that prevent unrelated context transfer. Role transitions must also be explicit: a tool-like scheduling assistant should not silently become a behavioral coach, and a caregiver-like system should not become a surveillance system (§7.3.2).

What this paper does not have

No experimental results. It proposes a framework and a research agenda; no model is trained and nothing is measured. No claim of the form "Combodied design raises accuracy by X%" appears anywhere in it.

CombodiedBench does not exist yet. §6.4 proposes design requirements and module names; there is no data and no result. §6.1 says as much itself — existing public resources evaluate isolated capabilities, and none covers the full loop under risk.

A PWM is not a new estimator. The paper calls it "an organizing abstraction rather than a single newly claimed estimator," and states that empirical validity must be established separately for each target, horizon, population, and authority level (§4.3). The causal machinery likewise depends on a defensible causal graph plus assumptions such as consistency, positivity, and control of time-varying confounding (§4.2).

Edge deployment is not automatic safety. Local storage does not by itself guarantee privacy, meaningful consent, or agency preservation; device compromise, loss, shared-device use, and synchronization conflicts all threaten the authoritative personal state (§5.2.3). Read it, then, not as a technique to ship next week but as a design-review checklist and a vocabulary for evaluation criteria.

Using this on the job

Who touches it, and when. Product designers on personal AI, health support, learning support, eldercare, and workplace assistants, plus architects of agent platforms. Three moments: (1) the design review where cross-session memory gets added, (2) the decision to introduce proactive notifications or nudges, (3) permission design for high-impact delegation — booking, sending, paying.

What actually becomes an artifact. The highest-yield section is the event-evidence record schema in §3.9, which converts directly into a table definition. Keep three distinct times (event occurrence / entry / receipt) in separate columns. Retain source, device type, sampling mode, calibration and model version, and signal-quality indicators. Flag observed fields separately from inferred fields. Do not discard contradictory evidence. Carry sensitivity, consent basis, retention, correction history, and deletion conditions. For external integration the paper names HL7 FHIR / SMART on FHIR and the OMOP common data model (§3.8); for evaluation references it points to LongMemEval, MemoryBank, PHIA, Health-LLM, and iOSWorld — each with the caveat that these test isolated capabilities (§6.1).

The mistakes that turn into incidents.

1. Mixing observations and inferences in one table. The moment you do, there is nowhere for a user correction to land. This is why the paper requires that "the person must be able to distinguish what they explicitly said from what the system inferred" (§3.1)
2. Using model confidence as grounds to intervene. Statistical confidence, sensor reliability, causal validity, and permission to act are four different things (§3.9). Gate the admissible set independently of confidence
3. Concluding causation from a metric that moved in an A/B test. Motivation, hidden context, and selective engagement all confound the result (§4.2, §8.4)
4. Putting engagement in the KPI. The paper calls this business-model misalignment: a system optimized for retention or advertising may learn to increase dependence and emotional intensity instead of wellbeing (§8.1)
5. Ignoring timing asymmetry. An agent that knows when someone is lonely, tired, anxious, or cognitively overloaded can pick moments of heightened susceptibility. The paper asks for auditability on high-impact recommendations (§8.1)

One question worth asking in review. "Does this feature grow the user's capability or substitute for it — and how would we know?" If you cannot measure it, the feature is not finished being designed.

Takeaways

• The claim is not "add features" but reorganize perception, memory, prediction, intervention, and evaluation around the evolving person (§1)
• The person's next state is not determined by the agent's action alone (Eq. 1). That non-determinism is the fork in the road
• A PWM returns scenario-conditioned trajectory distributions but authorizes nothing; authorization belongs to the admissible set (Eqs. 2–3)
• The success criterion gains eight agency-preservation metrics, and in high-stakes settings averages cannot offset severe failures (§6.3)
• It is a position paper with no experiments and no benchmark numbers. Read it as design-review vocabulary

Related reading

• LLM Agents Basics — the structure this paper proposes to reorient, before the reorientation
• The Personalization Mirage — measured evidence of what happens when observation and inference are not separated
• Metis: A Memory Foundation Model — the other side of longitudinal memory, built as a model