Paper Explained: Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation — Borrow Someone Else's Skill, Rewrite It for Your Own Harness

RASO retrieves agent skills from a large public corpus and rewrites them in the vocabulary of the target execution harness before using them. With a rollout-free initialization stage and a failure-driven update stage, it beat retrieval-free optimizers across four benchmarks and two models. A from-scratch walkthrough of the paper.

Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation

A pile of cookbooks and your own kitchen

When you're handed a dish you've never made, you look up someone's recipe first. But that recipe may assume an oven you don't have. So you translate "20 minutes in the oven" into "covered pan, low heat," rewriting it in terms of the tools in your own kitchen before you start.

The paper covered here, "Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation" (Chu et al., KAIST, Korea University and Meta AI, arXiv:2609.38024, posted September 29, 2026), brings exactly this find-translate-rewrite habit to building skills for AI agents.

Here is the abstract in brief. An agent skill is a reusable, actionable natural-language artifact that guides an agent to perform a task well under a given execution environment, or harness. Work on skill optimization has led to millions of publicly shared skills spanning many tasks, domains and harnesses. Yet existing optimization methods largely ignore this accumulated knowledge and rely only on expensive agent rollouts to refine a skill for the target task. The authors propose RASO (Retrieval-Augmented Skill Optimization), which uses an external skill corpus as prior knowledge throughout optimization. RASO retrieves relevant knowledge from existing skills and, through Cross-Harness Adaptation, adapts it to the target task and harness while accounting for mismatches in both domain and harness. It has two stages: RASI builds a knowledge-grounded initial skill without any rollouts, and RASU iteratively refines the skill by retrieving external knowledge guided by execution feedback. Across four agent benchmarks and two models, RASO consistently outperformed baselines that lack retrieval-augmented initialization and updating.

What are "skills" and "harnesses"?

LLM agents run inside an execution harness that defines which tools exist, which files they can touch and how they are scored (§1). Performance depends not just on what the model knows but on the procedure: what to do, in what order, in this particular environment. A skill is that procedure written down in natural language. Because it is text, the paper notes, it can be inspected and audited by people and carried across models unchanged (§1).

The catch is that existing automatic optimizers (TextGrad, GEPA, SkillOpt, WikiSkill and others) learn almost entirely from their own rollouts (§2). Rollouts are costly, and they can't teach procedures that lie outside what the agent can infer from its own failures. Meanwhile, public repositories already hold huge numbers of skills, but those were written for other domains and other harnesses. According to Figure 1, 92.9% of the skills retrieved for SpreadsheetBench come from a different harness. Someone else's recipe needs translating before it's useful.

The problem, as an equation

The paper frames optimization as follows (§3):

In words, keep the model and the harness fixed, change only the skill text, and pick the skill that earns the highest average score on the validation tasks.

$\mathcal{M}$ is a frozen language model, $h$ the harness, $x$ a single task and $s$ the skill text. $h(\mathcal{M},x,s)$ is the trajectory you get by running the model with that skill in the environment, and $r$ scores it between 0 and 1. In plain words: keep the model and the environment fixed, vary only the written instructions, and pick the instructions with the best average score on the validation tasks. Unusually, the paper counts writing the very first version of the skill as part of the optimization problem too.