Paper Explained: What Makes Good Agentic Data? The ACE Lens

A survey that recasts agentic training data as a four-part object (environment, task, interaction, verifier) and reframes generation as constrained distribution design: admit on Accuracy, place mass by Complexity, spread coverage with divErsity.

What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents

The problem the paper takes on

The original title is "What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents" (arXiv:2608.27260, 27 August 2026). It is a survey from Huawei, Shanghai Jiao Tong University and collaborators — not a paper that runs new experiments, but one that reorganises the field of agentic data generation itself.

The abstract argues as follows. LLM agents increasingly learn how to deal with external environments from generated interaction data. That generation has to keep environments, tasks, interactions and success signals mutually consistent while producing experience that is useful rather than merely abundant. Existing work, though, is organised by domain and evaluated heterogeneously, which hides the shared generation mechanisms and blurs the line between constructing candidates and verifying or selecting them. The paper therefore proposes a two-level framework. First, it represents agentic data as a common factorized object $(E,q,\tau,v)$ and classifies pipelines by which factor anchors the rest. Second, it formulates generation as constrained distribution design through the Accuracy–Complexity–divErsity (ACE) lens: accuracy fixes the feasible support of valid data; within that support, complexity decides where learning mass sits relative to a declared learner; and diversity controls coverage and redundancy.

An analogy: writing course material

Teaching an agent is like writing a problem set — except four things have to be in place at once. A lab (a place with tools, state and rules where you can actually do something), a task that is meaningful and achievable inside that lab, a worked solution that really runs there, and a marking scheme.

Break any one of them and the whole thing collapses. A task calling for a reagent that does not exist; an experiment that produces something else when you follow the steps; a marking scheme that accepts a wrong answer. The awkward part is that every one of these can come out looking perfectly plausible. The paper keeps returning to this point: individually reasonable components can drift apart when they are generated separately (§4.1).

The four-part data object

The paper models interaction as a POMDP (partially observable Markov decision process) (§2.1). The policy acts on the observable history, not on latent state.

In words: the next action $a_t$ is chosen using only $h_t$, the record of what the agent has seen and done so far. The environment may keep hidden state of its own, and that split between latent state and observation is exactly what separates agentic data from ordinary instruction-tuning data.

On that basis, the common form of a data point is (§2.3–2.4):

$E$ is the environment specification (tool schemas, database or simulator state, transition rules, permissions, and the observation interface that decides how much is exposed). $q$ is the task signal — what should be achieved, under which constraints. $\tau$ is the realized interaction, the sequence of actions and observations. $v$ is an optional verifier: the success check or reward. For SFT, storing $E$ and a fixed $\tau$ is usually enough; for RL and environment-based evaluation, $E$ must be able to support fresh interactions.

The factorization pays off because it makes the targets of ACE explicit. Accuracy is about consistency between the four factors, complexity about the burden their configuration induces, diversity about their non-redundant coverage.

ACE as constrained distribution design

The central formulation is §3.4. Of a generated batch, the subset that passes checks on environment consistency, task feasibility and trajectory validity is written $\mathcal{B}_A$. The paper's position is that accuracy is an admission condition, not a benefit other properties can compensate for. Difficult or unusual does not get broken data through the gate.

Within that admitted set, complexity is calibrated rather than maximised. The paper writes $C_z(d)$ for the difficulty of an instance under a declared learner and execution configuration $z$, where $z$ covers the model, the scaffold, the available tools, the verifier and the inference budget. Difficulty, in other words, is not a property of the data alone. Diversity $D(\mathcal{B}_A)$ is defined at the level of a batch, not a sample.

The three are asymmetric, and that is the point (§1). Difficulty or variation cannot make up for invalidity. Data that is uniformly valid but trivial or repetitive, meanwhile, adds little learning value. Hence the ordering: admit first, then place, then spread.

One more caveat: ACE is not a partition of the literature into three method families but a lens that views the same pipeline from three directions (§3.4). A verified blueprint can raise accuracy, lengthen the dependency structure and enable controlled recombination all at once; a failure-driven generator can target learner-relative complexity while narrowing domain coverage. Expect one mechanism to move several axes at the same time.