Paper Walkthrough: Training Agents to Evolve with Their Harness

Why compact models break when you rewrite prompts and tool definitions every week, and what the team behind Taobao Live's AI avatar streamers does about it. A ground-up walkthrough of Harness-Aware Training, from intuition to equations to measured numbers and stated limits.

Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report

The new hire who memorised the chapter headings

There are two ways a new hire learns a job. One is to memorise names and places: "returns question → chapter 3 of the returns manual." The other is to read what the manual actually says and decide from the content. The first gets you productive faster. But the day the manager reissues the manual with reordered chapters and renamed headings, only the first hire freezes. The work is identical — only the labels they were leaning on are gone.

The same thing happens to LLM agents. That is the subject of "Training Agents to Evolve with Their Harness: TaoLive Digital Avatar Agent Technical Report" (arXiv:2608.15763, TaoLive AIGC LLM Team), a report from the team running the AI avatar streamers on Taobao Live.

The abstract in short: AI digital avatar streamers must answer product questions, engage viewers and execute marketing strategies in real time, demanding low latency, frequent strategy updates, and answers that are both accurate and effective. An Evolvable Harness — Skills, Hooks, prompts and tools updatable independently of model weights — enables fast iteration but exposes a trade-off: large models adapt zero-shot yet are too slow, while compact models meet the latency target but overfit to a fixed Harness configuration. The paper proposes Harness-Aware Training (HAT), whose key component is Harness-State Augmentation (HSA): task-preserving transformations applied to Skill identifiers and content, tool schemas, prompt structure and Hook functions. Training runs in three stages — HSA-SFT, General On-Policy Distillation, HSA-RL. Across four evaluation sets, HAT reaches 94.8 on Live-Stream QA (base 80.3; strongest general LLM 93.0) and 94.6 on Harness-Variant QA (base 75.4). Where Fixed-Harness SFT drops IFEval 7.7 points below the base model, HAT avoids the regression and reaches 83.5. On a single NVIDIA H20 the optimised system delivers P50 3.4 s and P95 8.1 s, and the deployment yields positive online A/B results for GMV and item-page views in Taobao Live.

What the system actually does

To make the target concrete (§2.1): the avatar's text response goes through TTS, is synchronised with an audio-driven avatar generation model, and is broadcast to the whole live room. In other words the output is read aloud in public, so a wrong fact or an unsupported promise becomes a broadcast incident. Inputs combine the viewer's message with the host's dialogue history, product metadata, action link IDs and the room's real-time state. The internal traffic summary puts the intent mix at roughly 46% product Q&A, 19% casual chat, 16% clarification follow-up, 7% after-sales, 4% discount and promotion inquiries, and about 2% each for presentation-order changes and FAQ replies (Table 1).

A request passes five runtime stages: load and normalise context, assemble the prompt, run the agent loop (load Skills, issue tool calls, propose a final reply), apply Hook checks, then strip internal reasoning fields and dispatch (§2.4).

What a Harness is

The Harness is the configuration that sits outside the model and governs its behaviour. At runtime (§2.2) it has four parts.

• Skills: dynamically loaded behavioural modules for Q&A strategy, tool choice and reply style — versioned Markdown with YAML metadata
• System Prompt Pipeline: assembles instructions on the fly from product specifications, retrieved FAQ, room status, global boundaries and the descriptions of active Skills
• Hooks: lifecycle checks that validate inputs, parameters and output format, triggering correction or retry
• Tool Registry: connects the agent to product, inventory and commerce services; marketing interfaces are discovered dynamically over MCP

The paper bundles these into one state and writes the policy as $\pi_\theta(a \mid x, h)$, where $x$ is the user input plus business context and $h$ is the Harness state governing behaviour at inference time (§3.1).

— which says that the Harness state $h$ is the four-tuple of active Skills $\mathcal{S}$, tool registry $\mathcal{T}$, assembled system prompt $\mathcal{P}$ and Hooks $\mathcal{K}$. That is all equation (1) claims; the design decision behind it matters more: these four are versioned independently of model weights, so a new Skill or tool ships as a config change rather than a retraining cycle. For what a Skill really is, see why Agent Skills work and where they stop working.

Improving quality without touching weights — and where that stops

What makes the paper unusually honest is that it first measures the loop where nothing is trained and only the Harness is edited. Harness Evolution is human-in-the-loop: AI clusters failures and proposes edits, a developer confirms, evaluation runs, a human decides. Measured on a 482-item human-annotated dev set (§2.3, Appendix B.2):

Evolution 1 bought a large accuracy jump while refusal over-triggered and Effectiveness fell; Evolution 2 recovered both and became the selected checkpoint. Evolutions 3–4 — the "just add more rules" direction — made both metrics worse, because long-tail rules interact across Skills, Hooks and global instructions. The paper uses this as the motivation for configuration diversity during training.

The dilemma: fast models are rigid, flexible models are slow

If the Harness moves weekly, the policy lives on a changing execution environment. A strong zero-shot model can read the changes and adapt, but it is slow: DeepSeek-V4-flash has a median end-to-end latency above 11 seconds (§5.5 records P50 11.210 s, P95 21.191 s, and only 71% of requests inside 15 s). That does not work as live conversation.

So you reach for the compact Qwen3.6-35B-A3B — whose zero-shot accuracy misses the industrial bar, which forces domain training. And training on a single configuration produces exactly the new hire from the opening. The paper's term is surface-form overfitting: the model memorises specific skill names, tool names and prompt templates instead of interpreting the instructions it is given (§1).

Telling memorisation from understanding

If "overfitting" still feels abstract, play with the figure below: raise the polynomial degree and training error keeps falling while test error turns upward past a point.

Here, read the horizontal axis not as "model capacity" but as "degree of fit to the training-time Harness." Fixed-Harness SFT gains 9.2 points in-distribution while losing 4.6 points of Prompt Robustness and 7.7 points of IFEval prompt-level accuracy (§5.3). The gain and the damage share a cause.