Paper Explained: Co-Evolution in Agentic Systems — Three Stages Toward Self-Directed Evolution

Why do agents that are supposed to keep improving after deployment hit a ceiling? A ground-up walkthrough of a survey that organises the field into three stages — evolving peers, evolving environments, and an evolving evolution mechanism — with the defining equations, representative methods, and the open problems in evaluation and safety.

Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design

Stop running and the world leaves you behind

In Through the Looking-Glass, the Red Queen tells Alice that here you must run as fast as you can just to stay in the same place. Biologists borrowed the line for the Red Queen effect: predators get faster, so prey get faster, and the chase never ends.

This survey brings that idea to AI agents. The premise is simple. Agentic systems are expected to keep getting better after they are deployed, yet the dominant form of self-evolution moves only one side of the relationship (§1).

The ceiling that self-evolution runs into

Self-evolution means an agent keeps updating itself from experience, feedback, and failure without a human in the loop. The common shape is single-entity self-evolution — fine-tuning the model backbone, rewriting memory, adding skills — all of it happening inside one agent (§1).

The problem the paper names is that those updates sit inside a static learning context. Fixed tasks, fixed feedback rules, fixed counterparts. Solving the same benchmark over and over, sparring with the same opponent, leaving the reward scheme exactly as a human wrote it — in every case the learning context has stopped moving. When only one side adapts, the headroom eventually runs out. Sustained progress requires mutual adaptation rather than adaptation on only one side — that is the Red Queen effect restated for agents (§1).

Defining the agentic system first

The paper refuses to leave this vague (§2.1). An agentic system $S=(A,E)$ consists of an agent collective $A$ and an environment $E$. Each individual agent decomposes as:

Put in words: an agent ($a_i$) is nothing more than a brain plus the kit strapped to it, and the kit is part of the definition.

Here $m_i$ is the model backbone (the LLM or policy itself) and $h_i$ is the harness — memory, tools, skills, prompts, workflows. In everyday terms, $m_i$ is what moves when you train, and $h_i$ is what you touch as code: prompt templates, tool definitions, the memory schema. Two agents can share a backbone and still count as distinct if their harnesses give them different roles or objectives. "The agent evolved" is then defined as:

In plain terms: either the brain or the equipment changed, and that counts as evolution. $\Delta$ reads as "how much this changed since last time", $\iff$ as "the two sides mean the same thing", and $\lor$ as "or" — which says that training is not the only route in: rewriting a single line of a prompt already qualifies under this definition.

Agents are also not an unordered set. They come with a structure $\Pi$ encoding roles, communication topology, and division of labour, written $A=(\{a_1,\dots,a_n\},\Pi)$. A change to $\Pi$ alone already means $A$ evolved. The environment $E$ is everything outside $A$, and $\tau^t$ is the accumulated history of thought–action–observation cycles. If the basic anatomy of an agent is unfamiliar, LLM agents from the ground up makes the rest of this easier.

Co-evolution requires mutual pressure, not just contact

Call the mechanism that drives evolution $\Omega$, so that $S^{t+1}=\Omega(S^t,\tau^t)$. $\Omega$ is the bundle of rules that specifies what can evolve, when evolution triggers, how variants are produced, where it happens, and how its quality is judged (§2.2).

Against that backdrop the paper defines co-evolution as a setting where at least two evolving units jointly adapt and continually reshape each other's further evolution, rather than merely exchanging information or interacting. Talking to each other is not enough. One side getting stronger has to change the problem the other side faces.

There is a quick test. Freeze one side, rerun the same experiment, and see whether the result changes. If it does not, what you have is interaction, not co-evolution. That line is exactly what lets the survey exclude most multi-agent work from its scope. The taxonomy then orders the remaining literature by how far the boundary of evolutionary freedom expands (§2.3).

Stage 1: your peers move, so you have to move

Agent–Agent Co-Evolution (§3) holds the environment fixed and lets the evolution of peers or opponents drive your own.

Adversarial (§3.1) means each side is rewarded for defeating the other. The lineage runs back through GANs and RARL, which trains a control policy against an adversary that adds destabilising forces. The current centre of gravity is LLM safety: an attacker generating jailbreak prompts and a defender learning to refuse. A representative design deliberately makes already-found attacks stop working, so the attacker keeps discovering new attack types instead of collapsing onto a few modes (§3.1.1). Pressure can also come from many sides at once: league training — a main agent facing exploiters that target its weaknesses plus past versions that prevent forgetting — is the classical example (§3.1.2).

Collaborative (§3.2) means a shared goal. In the parallel form, the striking idea is drawing reward from the collaboration process itself — whether a turn corrected or persuaded later responses. The role-differentiated form starts from a produce-and-revise pair, including designs that reward a critique agent only when its suggestions actually improve the partner's next response. Evolving organisations (§3.3) goes one level up: when execution traces reveal a mismatch, the team structure $\Pi$ itself gets rebuilt.

Stage 2: let the environment move too

In Agent–Environment Co-Evolution (§4) the environment adapts. The paper classifies methods by which environmental change primarily drives further agent evolution.

Task space (§4.1) is about what the agent solves next. One branch keeps the pool fixed and adapts exposure (curricula weighted by learning progress); the other generates new tasks — calibrating difficulty to the solver's success rate, converting failed rollouts into tasks, pairing each task with a verification function. A small but sharp line is drawn here: even when a task is produced by an agent, it counts as environment evolution, because that agent is not a peer competing or collaborating inside the same task (§4.1.2).

Feedback space (§4.2) is about how the system evaluates, rewards, and diagnoses. Methods train a policy under a candidate reward function and rewrite that function from its failures, or evolve unit tests and surrogate verifiers from execution failures. A notable pattern adds consistency as a gate: update reward parameters only when several critics agree on a ranking, or update a diagnostic critic only when acting on its advice actually improved the policy. The groundwork is continuous with instruction tuning and RLHF; the difference is treating the reward model as a co-evolving counterpart rather than a fixed artefact.

Interaction space (§4.3) is the world the agent acts in: generating levels that target its weaknesses, widening simulation randomisation as the policy improves. It stretches to the rules governing agent populations — learning a tax policy, evolving social norms — and to replacing the real environment with a co-trained world model.

Stage 3: make the mechanism itself evolvable

Meta Co-Evolution (§5) stops treating $\Omega$ as a human-designed constant. A self-generated revision process $\Gamma^t$ is introduced:

The first equation rewrites how the system evolves; the second advances the system using the rewritten rules. Read it in words: the system first edits its own rulebook, then plays the next round by the edited book. $\Gamma^t$ is the editor, and it works from three inputs — the system as it stands ($S^t$), the rules currently in force ($\Omega^t$), and everything that has happened so far ($\tau^t$). $\Omega$ decomposes into five decisions — what to evolve, when (after failures, plateaus, or distribution shifts), how, where (sandbox, simulator, or the real world), and how to evaluate (§5.1).

The motivation comes from what Stages 1 and 2 look like empirically: the paper's summary of prior results says those stages improve performance in most settings, but the gains get smaller as evolution approaches a plateau (§5.1). The authors are candid that almost nothing yet qualifies for Stage 3 — methods that evolve mutation prompts, self-modification machinery, or the memory architecture are single-entity precursors, with no lower-level co-evolving system underneath them (§5.2).

What co-evolution is not

Appendix A works through the adjacent concepts one at a time, and this is arguably the most immediately useful part of the paper.

• Multi-agent interaction: communicating or competing without changing yourself is not it. Persistent updates have to land on multiple sides.
• Harness engineering: designing memory and tools is not evolution, and even when model and harness both change, that is still one evolving agent.
• Self-play: interacting with copies or earlier versions of yourself can support self-evolution while only the current learner changes.

How to read Figure 4 without over-reading it

The survey runs no new experiments. Figure 4 summarises results reported by prior work, and Appendix C is explicit that raw scores are not pooled, because the papers use different models, benchmarks, and metrics. A paper is included only when it reports a matched pair — a static counterpart and an evolving one under the same backbone and evaluation setting. The trajectories include values approximately digitised from published plots, and the shaded band reflects cross-paper heterogeneity, not a confidence interval. It visualises a tendency, not a performance guarantee.

Limitations the authors state

Two, in their own Limitations section. First, meta co-evolution is still at an early stage: only limited work meets the Stage 3 definition, so much of the discussion leans on single-entity meta-evolution as a precursor. Second, safety and governance are not operationalised: the failure modes specific to co-evolving systems are named but treated as desiderata. And being a survey, it contributes no new experimental evidence of its own.

How this is used on the job

Who and when: teams running agent platforms or RL training pipelines, plus the evaluation and red-team side. The taxonomy is most useful as a map when you have already added a self-improvement loop and the gains have stalled — it tells you which knob to make adaptive next.

What you actually touch: the opponent pool (a league, snapshots of past versions), the version of the reward model or verifier, the difficulty target of a task generator (calibrated against the solver's success rate), and a held-out evaluator you never train against. For Stage 1 the first move is "stop freezing the counterpart"; for Stage 2, pick exactly one of tasks, rewards, or world to make adaptive.

The failure modes in §6 map straight onto production incidents. (1) Evaluator exploitation — co-evolve the evaluator and the score can rise while behaviour becomes exploitative. (2) Partner overfitting — strong against the counterpart you trained with, weak against unseen partners. (3) Diversity collapse — the attacker side degenerates into a few modes and the defender becomes robust only to those. The countermeasure is pairing fixed benchmarks with process-level testing: historical cross-play, component ablations, and re-measurement with held-out evaluators (§6). The most useful question in a design review is "would this improvement have shown up with the counterpart frozen?" — if you cannot answer it, build that control condition first. The evaluation groundwork is shared with overfitting and evaluation.

On governance, the paper points to sandboxed deployment, continuous monitoring, rollback to verified states, and explicit human intervention points (§6). Define what a "verified state" is before the system starts co-evolving, or you will have nowhere to roll back to.

Takeaways

• Single-entity self-evolution is bounded by a static learning context; getting past it means letting the other side move too.
• The three stages track how much is allowed to change: peers and opponents → tasks, feedback, and worlds → the evolution mechanism itself.
• The test is that both sides change persistently and reshape each other's later evolution. Plain interaction and self-play do not qualify.
• The open problems are evaluation, scaling, and safety — and evaluator exploitation and partner overfitting get harder to see precisely as the score goes up.