Paper Walkthrough: DART-SD — Training Tool-Calling Agents Without Flattening the Diamond

When a multi-turn tool-calling task has order-independent sub-goals, the set of correct solutions spreads out into a diamond lattice. This walkthrough explains why whole-trajectory imitation crushes that structure, and how locating the first point where a rollout leaves recoverable territory — and supervising only what comes after it — changes the picture.

DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents

When there is no single right order

Think about arranging a business trip. Book the flight, book the hotel, reserve the meeting room. These three can be done in any order. Flight first or hotel first, once all three are done you are in the same state: ready to travel.

Yet the dominant way we train agents is to have them copy the expert's steps one at a time. If the expert happened to go flight → hotel → room, then a student that goes hotel → flight → room gets penalized the moment it deviates — not because the work was wrong, but because it did not match.

The paper this article walks through is "DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents" (arXiv:2608.18524, ByteDance and the University of Science and Technology of China, published 2026-08-19).

Its abstract, restated: giving LLMs multi-turn tool-calling ability is essential for autonomous agents, but progress is fundamentally limited by the reliance on full-length trajectory imitation. For tasks with multiple order-independent sub-goals, the optimal solution space forms a vast combinatorial diamond lattice. Forcing that rich topology into monolithic trajectories causes a severe topological collapse, indiscriminately penalizing valid alternative explorations and badly degrading policy diversity. The authors therefore shift the paradigm from global forcing to topology-guided localized correction. DART-SD models execution as a converging Interaction-State Transition Graph (ISTG), identifies the Critical Topological Breakpoint (CTB) during the student's own rollouts, retrieves success-supported recovery references, and computes the training loss exclusively on the generated recovery steps, strictly protecting the valid reasoning prefix from destructive gradient updates.

What the diamond lattice is

With $k$ order-independent sub-goals there are $k!$ orderings. But those orderings reconverge: "flight and hotel both booked" is a single state no matter which came first.

So the solution space has one source, one sink, and a huge bulge in between — pointed at the top and bottom, wide in the middle. That is the diamond lattice the paper names (§1).

Seen through that lens, the failure modes of existing methods line up:

• SFT / behavior cloning: one teacher trajectory is declared correct and an indiscriminate global loss is applied to every token, overwriting the student's own valid exploratory steps (§1, §2).
• Standard RL such as GRPO: the reward arrives only at the end, so it is spread uniformly across all intermediate tool calls. Valid steps that happened to sit inside a failed trajectory get punished along with the fatal one (§1).
• Hindsight-based scaffolding: still treats multi-turn interaction as a strict linear sequence, so it cannot tell a fatal error apart from a harmless detour (§1).

What is being ignored, the paper argues, is the graph structure of state transitions. Distinct trajectories intersect at shared intermediate states all the time; a linear view simply cannot see those intersections.

Mechanism 1: nodes are accumulated state, not actions

DART-SD's first move is to rewrite execution as a graph of states rather than a sequence of actions (§3.1). The key is defining a node as what has been accumulated so far. If nodes were actions, a different order would be a different object; if nodes are accumulated contents, different orders land in the same place.

Information atoms. Useful facts extracted from tool responses are normalized into a task-specific set $\mathcal{K}_x$. Semantically equivalent responses share an atom; non-informative responses contribute none. This runs in two stages. A deterministic stage parses each response into fields and calls it non-informative only when every field is a status signal or an empty/placeholder value — a payload retaining a single substantive value still counts as informative, while an error or not-found message does not. A semantic stage then assigns atoms to the surviving candidates, judging all of a task's candidates jointly rather than labeling them independently.

Here $e$ is one tool call, $\operatorname{tl}(e)$ is which tool it used, and $\bar{o}(e)$ is the canonicalized response. Equation (1), in words, maps a (tool, response) pair to at most one atom. A response carrying no information maps to the empty set — nothing was gained.

$B_t$ is the tool call (or bundle of concurrent calls) executed at step $t$, and $I_t$ is everything learned up to that point. Equation (2) is the rule which says: take only what is newly learned this step and add it to what you already hold. Re-fetching a known fact leaves $\Delta I_t$ empty.

Two consequences follow. First, the same fact returned as free text by one tool and as a structured record by another yields one atom, so the two acquisition paths reconverge at the same state instead of forking. Second, non-informative responses never receive an identifier of their own, so they cannot manufacture states no evidence supports. The paper notes that residual judgement errors therefore tend to omit an atom rather than invent one, leaving the graph sparser but never fabricating states or reachability (§3.1).