2026-08-13·FREE·PAPER·11 min read

Paper Deep-Dive: Frontis-MA1 — Training the AI That Builds AI: One Step Toward Recursive Self-Improvement in ML Engineering

Instead of training a model to solve tasks, train the operations that improve solutions — then run evolutionary search with those trained operations. A walkthrough of the OpenMLE stack and the 35B Frontis-MA1 model, straight from the paper: what's behind the 39.39%→71.21% jump on MLE-Bench Lite, how the search got cheaper and better at once, and the limits the authors themselves spell out.

Key points

AI4AI puts AI systems to work on building and improving AI, and its ambitious endpoint is recursive self-improvement (RSI).
OpenMLE is a three-layer stack of Gym, ERL and Evo, and the 35B-parameter model Frontis-MA1 is trained on top of it.
The four operators Draft, Improve, Debug and Crossover become a shared vocabulary between training and inference, a coupling the paper calls meta-evolution.
OpenMLE-Gym is a suite of 5,758 executable, scoreable task environments, with full task data released for only 1,415 of them.
On the 22 tasks of MLE-Bench Lite, under the same harness Frontis-MA1-35B reaches 60.61% Medal Average, up from the base model's 39.39%.

#agents #recursive-self-improvement #evolutionary-search #reinforcement-learning #mle

ModalitytextTaskagents

Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering

Primary source — what this article is built on

undefined2026-07-30→undefined2026-08-13same month

Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning EngineeringJunlin Yang, Che Jiang, Yu Fu et al. · 2026-07-30 · v1arXiv:2607.28568Paper page·PDF

undefined

Recursive self-improvement (RSI) requires AI systems that improve the process of building AI (i.e., AI4AI); machine learning engineering (MLE) offers a concrete, executable testbed for studying this capability. We introduce OpenMLE, an open full-stack system for RSI research in MLE, spanning verifiable task environments with execution feedback (OpenMLE-Gym), operator learning (OpenMLE-RL), and long-horizon search (OpenMLE-Evo). On this stack we post-train Frontis-MA1 (35B) as a meta-evolution agent for MLE, aligning post-training and inference around four atomic program-evolution operators (Draft, Improve, Debug, Crossover): the same operators are trained via execution-grounded SFT and RL on data deduplicated against all evaluation benchmarks, then composed into long-horizon search, coupling learning and evolution in a single loop. On MLE-Bench Lite under a 12-hour per-task budget on one RTX 4090 capped at 12 GB VRAM, Frontis-MA1 (35B) improves Medal Average from 39.39% to 60.61% over its base model with OpenMLE-Evo, and reaches 71.21% with OpenMLE-Evo-Max (benchmark-independent experience priors and asynchronous search), exceeding GPT-5.5 + Codex and approaching GPT-5.6 Sol and the 2.8T Kimi K3. On held-out NatureBench Lite, both components transfer: with the framework fixed, swapping in the trained model raises Match-SOTA from 50% to 70%; with the model fixed, swapping in OpenMLE-Evo raises it from 20% to 50%. We release the model weights and the full OpenMLE stack to enable reproducible research on executable AI4AI toward RSI. Code: https://github.com/FrontisAI/OpenRSI

A Blacksmith Reforging Their Own Hammer

A blacksmith who wants to get better has two options. One is to keep hammering iron and polish each piece. The other is to reforge the hammer itself — a better hammer lifts every piece made from then on.

In AI, the "better hammer" agenda is called AI4AI (AI for AI): putting AI systems to work on the job of building and improving AI. Its ambitious endpoint is recursive self-improvement (RSI) — a loop where each improved system goes on to improve the very process that produces its successors (§1).

This paper (from Horizon Research at Frontis.AI together with Tsinghua University and collaborators) grounds that grand theme in something concrete and executable: machine learning engineering (MLE). MLE is Kaggle-style work — inspect the data, write a model, run it, read the score, revise. Because every attempt returns a number, "did we actually improve?" can be verified without hand-waving, which makes MLE a convenient testbed for RSI research (§1).

There are two deliverables: an open three-layer stack called OpenMLE, and a 35B-parameter model trained on top of it, Frontis-MA1. The authors state they will release the model weights and the full stack — datasets, training and evaluation code, sandbox infrastructure, and the search harness (§1).

The Big Picture: Three Layers and Four "Moves"

OpenMLE splits into three layers. OpenMLE-Gym is a suite of 5,758 executable, scoreable task environments (§3). OpenMLE-ERL is execution-grounded SFT plus reinforcement learning (§4). OpenMLE-Evo is a long-horizon evolutionary search harness (§5).

The linchpin is a set of four atomic operators: Draft (write a solution from scratch), Improve (make an existing solution better), Debug (repair a broken one), and Crossover (fuse the best parts of two solutions) (§4.1). In prior work, a search harness (scaffold) would simply call a frozen model, and the model never learned anything from the search. In OpenMLE, these four operators become a shared vocabulary between training and inference: verified improvement steps produced by search become training data for exactly those operators, and the trained model in turn becomes the "variation engine" of the search. The paper calls this coupling meta-evolution, which is where the name Frontis-MA1 (Meta-evolution Agent 1) comes from (§1, §2).

The Problem in One Equation

A single search step can be written as (§2):

p_{t}\sim g_{\theta}(\cdot\mid\tau,\,a_{t},\,c_{t}),\qquad s_{t}=R_{\tau}(\mathcal{E}(p_{t},\tau))

(1)

In plain terms: given a task $\tau$ , a chosen operator $a_{t}$ (one of the four moves), and a context $c_{t}$ assembled from parent programs and their execution feedback, the model $g_{\theta}$ proposes a candidate program $p_{t}$ . The sandbox $\mathcal{E}$ executes it, and the task-specific evaluator $R_{\tau}$ returns a score $s_{t}$ . Said in words, the line is one lap of an ordinary working day: pick a move, look at what has already been tried, write some code, run it, read the number that comes back. Within a fixed budget, search hunts for the highest-scoring program. On the learning side, both SFT and RL collapse into a single weighted loss: raise the probability of programs whose execution scores are strong (§2).

Why You Must Not Only Dig at the Best Solution

Before diving into the layers, here is the intuition that runs through the whole paper. Evolutionary search keeps choosing which node (candidate solution) to expand next — and greedily extending only the current top-scoring branch starves the branches with headroom. The paper gives a concrete example from a whale-call detection task: a candidate that ranked only sixth by score but first by improvement-over-its-parent was picked up by a three-factor selection rule (quality, progress, novelty), and its descendant eventually reached validation AUC 0.99203 and held-out AUC 0.99386. Under score-only selection, that branch's selection probability was 10.47%; the three factors lifted it to 17.09%, and that difference mattered (§6.5).

Selection is turned into probabilities via a softmax over utility, with a temperature that controls how peaked the distribution is. Drag the temperature in the figure below to feel the greed-versus-exploration trade-off.

FIG 1Lower the temperature and the top candidate wins almost every time (greedy); raise it and selection spreads out (exploration). OpenMLE-Evo's parent selection feeds three factors — quality, progress, novelty — into exactly this kind of distribution

From here, let's open up the three layers of the stack, bottom to top.

OpenMLE-Gym: 5,758 Problems You Can Actually Run

Each task is an environment with five elements: a state made of the specification, public data, a hidden evaluator, and a resource budget; an action, which is the program the agent submits; a transition, which is sandboxed execution; an observation, a structured record of execution status, score, logs, and artifacts; and a reward, the score returned by the evaluator (§3.1). It is the familiar Gym-style contract from reinforcement learning, applied to MLE jobs that can run for hours.

Tasks come from three sources: 156 hand-picked anchors from papers and benchmarks, 3,362 tasks auto-generated from Kaggle datasets, and 2,240 converted from Kaggle competitions (§3.5). Because automated generation is noisy, every package passes an LLM-based quality filter that scores five dimensions — task validity, data sufficiency, raw-data usage, task complexity, and data quality — and only packages with a strict "recommended" verdict survive (§3.3). Competitions overlapping with MLE-Bench are excluded at construction time to protect evaluation integrity (§3.2). One honest caveat: due to licensing, full task data is released for only 1,415 of the 5,758 tasks; the rest ship as scripts only (§3.5, footnote).

A quietly important detail is the sandbox's six feedback modes — success, runtime error, missing code, missing submission, scoring failure, and timeout (§3.4). "The code doesn't run" and "the code runs but the solution is weak" are different failures, and being able to tell them apart underpins the quality of both training and search downstream.

OpenMLE-ERL: Training the Operations Themselves on Execution Results

Learning happens in two stages. First, execution-grounded SFT. Teacher models generate solutions, everything is executed, and score-based filtering assembles a corpus of 26,259 examples: 17,245 complete Draft solutions sampled independently, plus 9,014 trajectory steps extracted from evolutionary searches — keeping only the steps that causally contributed to the final solution (§4.2, Appendix B.1). Collection is budget-adaptive: a task stops once it hits its accepted-example quota or exhausts its execution budget.

Then reinforcement learning, where an MLE-specific problem appears: every task has a different metric with a different range (accuracy, log loss, RMSE, ...), so rewards are not comparable. The first step is normalizing scores into [0, 1] using a pair of bounds (§4.3):

r_{\mathrm{base}}(\tilde{s};b_{\mathrm{best}},b_{\mathrm{worst}})=\mathrm{clip}\!\left(\frac{\tilde{s}-b_{\mathrm{worst}}}{b_{\mathrm{best}}-b_{\mathrm{worst}}},\,0,\,1\right)^{\alpha}

(2)

In plain terms: convert the score $\tilde{s}$ (flipped so that larger is always better) into "where does it sit between the worst bound $b_{\mathrm{worst}}$ and the best bound $b_{\mathrm{best}}$ ?". The fraction is a position on a ruler, which says 0 if the program is no better than the worst bound, 1 if it matches the best; clip pins anything falling off either end back onto the ruler, and the exponent $\alpha$ raises that 0-to-1 position to a power, bending how steeply reward climbs across the band. The catch is that theoretical or leaderboard extrema are usually far wider than the score band the current policy can reach, so meaningfully different programs collapse to nearly identical rewards. OpenMLE therefore rebuilds adaptive bounds from each task's observed on-policy scores, preserving resolution where the policy currently lives (§4.3).

On top of the normalized reward comes an entropic advantage:

A_{i}^{\mathrm{ent}}\approx\frac{\exp(\beta\, r_{\mathrm{proc},i})}{\frac{1}{G-1}\sum_{j\neq i}\exp(\beta\, r_{\mathrm{proc},j})}-1

(3)

In plain terms: within a rollout group of $G$ candidates, the better a candidate ranks, the exponentially larger its learning signal ( $\beta$ is the concentration knob). The fraction weighs one candidate's exponentiated reward against the average of its groupmates', which says in words: how far above the room am I? A candidate that is merely average lands at zero once the $-1$ is applied, while a standout pulls away from that line fast. Since MLE is judged by the best program found, a barely-working submission must not receive the same positive reward as a top solution (§4.3).

On the engineering side, waiting for a synchronous batch means idling on the slowest sandbox job — some runs take hours — so OpenMLE feeds completed generation-and-execution groups to the trainer as they finish. Measured mean step time dropped from 97.0 to 50.8 minutes (Appendix B.4). The choice of which parent program to practice an operator on also matters: parents are sampled by a fitness score combining parent reward, child-reward variance, and a visit-based cooling term (Equation (3), §4.3).

OpenMLE-Evo: Driving the Search with Experience Cards

The inference-time harness builds on an AIRA-Evo-style population loop but redesigns how it uses evidence. Every time a node is evaluated, the harness writes an experience card — deterministic metadata on provenance, performance, execution outcome, and resource usage — and aggregates all cards into a task-global experience board tracking method families, their best candidates, underexplored directions, recurring failures, and score trends (§5.1).

Parent selection is the three-factor rule previewed earlier (§5.2):

U_{i}=\lambda_{s}\tilde{s}_{i}+\lambda_{\Delta}\widetilde{\Delta}_{i}+\lambda_{n}\nu_{i},\qquad P(i\mid\mathcal{I})=\frac{\exp(U_{i}/\tau)}{\sum_{j\in\mathcal{I}}\exp(U_{j}/\tau)}

(4)

In plain terms: candidate $i$ 's utility $U_{i}$ is a weighted sum of its normalized score $\tilde{s}_{i}$ (quality), its improvement over its parent $\widetilde{\Delta}_{i}$ (progress), and its method-family novelty $\nu_{i}$ ; a temperature- $\tau$ softmax turns utilities into selection probabilities. Put in words, the harness asks three questions of every candidate before it picks a parent — how good is it, how much did it gain on its own parent, and is anyone else already working this way? — and the temperature decides how strictly the winner of that scoring gets favored. This is exactly the moving figure from the beginning of the article.

Memory is handled frugally too. The original AIRA-Evo eagerly summarized every node's history with an LLM; OpenMLE-Evo synthesizes summaries on demand, only for nodes actually selected by an operation, and caches them (§5.3). Context is also operator-specific: Improve receives the node's vertical ancestor trace plus a horizontal set of sibling nodes; Debug retrieves past attempts sharing the same error signature; Crossover gets both parents' contexts plus a method-complementarity cue (§5.4).

FIG 2Greedy descent tends to stop in the nearest basin. Mixing progress and novelty into parent selection gives the search something like momentum, nudging it to try other basins — play with the momentum slider to feel the analogy

Results: Training Gains and Search Gains Add Up

Evaluation uses the 22 tasks of MLE-Bench Lite, with a modest budget of 12 hours per task on a single RTX 4090 capped at 12 GB VRAM, averaging three independent runs per configuration (§6.1). The headline numbers (§6.2, Table 1):

Effect of the model: under the identical OpenMLE-Evo harness, the Qwen3.6-35B-A3B base scores 39.39% Medal Average; Frontis-MA1-35B reaches 60.61% (Human Rank 0.5828 → 0.7647)
Effect of stronger search: with OpenMLE-Evo-Max (benchmark-independent experience priors plus asynchronous parallel search), 71.21% and Human Rank 0.8126 — above GPT-5.5 + Codex at 68.18%, approaching GPT-5.6 Sol and Kimi K3 at 72.73%
Replication on a second backbone: the 30B variant moves 34.85% → 53.03% → 66.67% in the same pattern
Harness comparison: with Frontis-MA1-35B held fixed, original AIRA-Evo gets 53.03% versus 60.61% for OpenMLE-Evo; for GLM-5.2, MiniMax M3 and others, OpenMLE-Evo also beats general-purpose scaffolds like Claude Code and Codex

The efficiency analysis is concrete (§6.5). Across 66 matched task-runs against original AIRA-Evo, total model tokens fell from 129.3M to 75.3M (−41.7%) while new-best validation updates rose from 229 to 246 — per million tokens, that's 1.77 → 3.27 (+84.3%). The fraction of Improve calls that set a new best doubled from 4.73% to 9.36%, and the 99th-percentile Improve prompt length shrank from 389.0K to 54.3K characters (−86.1%). This backs the claim that replacing serialized full histories with bounded, operator-conditioned context helps both cost and quality.

There is transfer evidence as well. On NatureBench Lite (10 tasks asking agents to match published Nature-family results), swapping in the trained model with the framework fixed raises Match-SOTA from 50% to 70%; swapping in OpenMLE-Evo with the model fixed raises it from 20% to 50% (§6.6). And trajectory analyses show that on a leaf-classification task, 85.0% of the validation gain came from late-stage Improve and Crossover operations — long search time is spent accumulating, not repeating (§6.3).

The Paper's Limits — RSI Is Not Here Yet

The authors devote a full section to this (§8). Five boundaries: (1) the reward is only the executed solution's final score, which cannot judge whether a research direction is promising; (2) the evolutionary harness and general coding agents remain separate, bounding what the model can initiate on its own; (3) the improvement target is external ML models, not the language model itself; (4) the evolutionary system itself stays fixed rather than being evolved; (5) the experience cards carry rich metadata, but parent selection uses only three hand-designed factors with fixed weights. Section 7 states plainly that this is not a claim of general, autonomous recursive self-improvement.

As a reader, add this: the transfer study covers 10 tasks, and the per-modality analysis works with small groups, so the breadth of both conclusions is limited (the paper concedes this in §6.6). Also keep in mind that MLE-Bench Lite is the 22-task reduced split of the full benchmark.

How This Is Used in the Field

This paper is dense with design lessons for people who build agent harnesses.

Who uses this, and when: engineers building LLM-agent platforms or AutoML-style products, whenever they design or refactor a search harness. The single biggest lesson: make the vocabulary of trained operations identical to the vocabulary the search invokes. If you roll your own, start by defining operators like Draft/Improve/Debug/Crossover as types, then align both your prompt templates and your training-data format to those types.
Parameters you will actually touch: the three selection weights (the paper's worked example uses Score/Gain/Novelty = 1.0/0.6/0.3) and the temperature $\tau$ ; the operator sampling ratio during RL (Draft 0.50 / Improve 0.17 / Debug 0.17 / Crossover 0.16, Appendix B.3); the per-task sandbox budget (12 GPU-hours here); the reward concentration $\beta$ .
Pitfalls that cause real accidents: (1) skipping deduplication between training data and evaluation benchmarks makes your numbers untrustworthy — the paper explicitly dedupes against all evaluation benchmarks. (2) Serializing the entire search history into every prompt explodes token costs — bounded operator-conditioned context accounted for the −41.7% difference. (3) Conflating "the run succeeded" with "the solution is good" — keep the six feedback modes separate, and weight rewards toward the top. (4) Normalizing heterogeneous metrics with theoretical extrema flattens all differences and stalls RL — you need adaptive bounds.
An interview-style question this paper answers: "What happens if a search tree always expands the highest-scoring node?" — complementary branches die off and Crossover loses its raw material, an answer this paper backs with evidence (§6.5).