Paper Deep-Dive: ABSeeker — Training Long-Horizon Search Agents by Grading Each Step Backward from the Answer

When an agent chains dozens of web searches, the only reward is whether the final answer was right — so every good move and every bad move along the way gets graded identically. We walk through ABC (Answer-Backtracked Credit Assignment), which recovers clues by working backward from the answer and scores every step against them, and ABSeeker, the 4B model trained with it — all from the paper itself.

ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment

The coach who only tells you won or lost

You finish a hundred-move chess game, and all your coach says is "win" or "loss." Could anyone improve under that regime? Won games still contain blunders; lost games still contain brilliancies. Praising (or scolding) every move uniformly tells you nothing about which move actually mattered.

Training web search agents sits in exactly that position. Long-horizon search agents like OpenAI's Deep Research or Tongyi DeepResearch reach an answer by stacking up many moves — search, read a page, revise the hypothesis, search again (§1). Yet conventional training, both supervised fine-tuning (SFT) and reinforcement learning (RL), has treated every step in a trajectory identically. Successful trajectories contain mistaken and redundant steps; failed trajectories contain the step that unearthed the decisive piece of evidence (§1).

This paper (arXiv:2608.05102, from a group at Shanghai Jiao Tong University) offers a simple and forceful answer to that credit assignment problem — how to distribute the praise and blame for a final outcome across individual steps. The idea: if you already know the answer, just walk the path backward from it.

The formal setup: the wall of sparse reward

Let's pin the problem down. Training data is a pair — a question $q$ and a verified gold answer $a^{*}$. The agent produces a trajectory over $T$ turns of interaction (§3.1).

$\tau$ (tau) is the whole trajectory, $s_t$ is the $t$-th step (that turn's reasoning, tool call, and environment response, taken together), and $a$ is the answer finally submitted. The equation, put in words, is just the record of it: a first move, a second move, on through the $T$-th, and then one answer handed in at the end. A trajectory is nothing more than a sequence of moves plus a final answer.

The conventional reward checks only that final answer.

One point for the entire trajectory if the answer is right, zero if it isn't. That's the whole signal — a rule which says: check the last thing the agent said, then hand that single verdict down over all hundred moves at once. Being sparse and coarse, it produces two failure modes (§3.1). First, the useful steps inside a failed trajectory — finding correct evidence, narrowing the candidates — never receive any positive signal at all. Second, the steps inside a successful trajectory that drew a wrong conclusion or discarded good evidence get reinforced right alongside the good ones.

The core idea: walk the path backward from the answer

The paper's starting point is a peculiar property of search tasks: once the correct answer is in hand, the task becomes naturally reversible (§1). Picture reopening a case after the culprit is known. Working back from the answer, you can pinpoint which interview was decisive and which testimony should have been chased.

The proposed method, ABC (Answer-Backtracked Credit Assignment), comes in two stages (§3).

1. Answer-Backtracked Clue Recovery: work backward from the gold answer to recover the set of intermediate clues — entities, facts, relations — that should have been discovered to solve the problem
2. Clue-Anchored Step Scoring: score every step against the recovered clues, converting the sparse 0/1 reward into a dense per-step reward

How clue recovery actually works

From each training question $(q, a^{*})$, the method builds a clue set $\mathcal{C} = \{c_1, \ldots, c_K\}$. Each $c_k$ is a verifiable piece of intermediate evidence linking the question to the answer — a particular entity, fact, attribute, or relation (§3.2).

What matters is that this recovery is not armchair generation. The recovery LLM runs as a ReAct loop with the same tools the forward-searching agent has — web search and page browsing — and travels from the answer toward the question, confirming evidence against real web pages as it goes. Only the clues that survive verification become scoring anchors (§3.2).

The paper's worked example (§3.2, Fig. 3): from a question built out of four constraints and the answer "CeraVe," the recovery model produces six clues — "ceramide" as the ingredient with clinical backing, "L'Oréal" as the acquiring company, "Eugène Schueller" as its founder and a member of the class of 1904, and so on. Together they form a verified chain of evidence connecting the question's constraints to the answer.

The rules for scoring each step

Once the clue set $\mathcal{C}$ exists, every step of every collected trajectory — successes and failures are both kept — gets scored. The scoring LLM receives the step's reasoning, tool call, and response, plus the original question and the clue set, and returns a score with its justification (§3.3). The rubric is strikingly compact (§3.3, Table 1).