Paper Walkthrough — J-Zero: Growing the Challenger, the Solver, and the Judge Together from Zero Data

A model that writes its own problems, solves them, and grades them — with no external data and no human labels. J-Zero's twist is that the grader learns too, which is what lets it keep improving past the two-iteration wall where prior methods stall.

J-Zero: Unified Challenger--Solver--Judge Co-Evolution from Zero Data

You cannot get better than your referee can see

Picture a small training hall. Practice needs three roles: someone who sets hard challenges, someone who works through them, and someone who scores the result.

If only the first two get stronger, something odd happens. The scorer's eye stops keeping up. A technique they once called sloppy now looks, to them, just like every other good technique. Every attempt gets the same mark, and the student loses the information about which way to adjust. Practice continues; progress does not.

The same thing happens in self-training language models. The paper's central observation is that a frozen Judge becomes the ceiling itself: it can only push the Solver toward preferences it has already internalized, so once the Solver saturates the distinctions the Judge can make, further training produces no signal (§1). J-Zero's answer is to train the scorer too. The name stands for "Judge co-adaptation from Zero data."

Problems you can check, and problems you cannot

Self-evolution research has mostly lived in verifiable domains — math and code, where the answer is unique. If there is a ground truth, grading is string comparison or code execution.

Then there are unverifiable domains. "Rewrite this proposal" has no single correct answer; quality is defined by human preference (§1). Here a Judge has to stand in for the verifier, and the ceiling problem shows up.

The prior work sorts into three groups (§2): methods relying on external ground-truth labels, methods relying on an external seed corpus, and fully data-free self-play. Even among the data-free frameworks, Absolute Zero got its reward from a code executor and R-Zero from majority voting over sampled answers. Both are cheap and hard to game, but both are largely confined to verifiable domains. The gap J-Zero targets is a setting where the evaluation signal is itself learned and keeps improving, in both kinds of domain.

The three players

• Challenger $C_{\theta_c}$ — writes problems; rewarded when the Solver does badly
• Solver $S_{\theta_s}$ — answers them; rewarded when the Judge scores it highly
• Judge $J_\phi$ — assigns a scalar score to a (task, response) pair

The Judge's raw output is squashed into $[0,1]$ by $\sigma(\cdot)$ before use (§3.1).

Put in words: the Solver's grade is the Judge's opinion, and nothing else. No answer key, no test suite. The whole loop's notion of "good" comes from another model, and this one line is where that dependency lives.

Here $x_i$ is the $i$-th task, $y_{i,j}$ the $j$-th response to it, and $r^S_{i,j}$ the resulting score between 0 and 1. $\sigma$ is the sigmoid, which compresses a raw score anywhere from $-\infty$ to $+\infty$ into that range.

A minimax game between the Challenger and the Solver

The two policies are trained adversarially with GRPO (§3.1).

The left half says "write problems the Solver cannot score well on"; the right says "score well on them." One player pushes the same Judge score down, the other pushes it up.

What the pair says in words is that nobody has to author a difficulty ladder. As the Solver improves, the same tasks stop earning the Challenger any reward, so the Challenger is pushed one notch harder. The curriculum tracks the Solver's current position on its own — that is the point of framing this as a minimax game rather than a fixed task set.

It is not strictly zero-sum, though. Maximizing difficulty alone pushes the Challenger toward near-duplicate problems and malformed output, so — following R-Zero — the reward combines difficulty, a repetition penalty, and a format check (§3.1). Difficulty is $1-\bar{r}_i$, where $\bar{r}_i$ is the mean score over the Solver's $M$ responses to task $x_i$. Repetition is measured by the pairwise distance $d_{pq}=1-\mathrm{BLEU}(x_p,x_q)$: near-identical tasks are grouped into clusters, and larger clusters are penalized more. The format check requires the task to be wrapped in <question> tags.

Put in words: the Challenger's score is how badly the Solver does, minus how much the task duplicates its siblings, floored at zero so it never goes negative. A pile of near-identical hard tasks would only make the Solver good at exactly one thing before it stalls, so the repetition term is what steers the Challenger toward differently hard tasks.

Choosing training problems by how much the scores scatter

Once the Challenger is updated it is frozen, and a larger pool of candidate tasks is sampled. The rule for picking which ones to train the Solver on is an unglamorous but load-bearing detail (§3.1).

Put in words: a task whose scores scatter is a good teaching example. One the Solver always scores well on has nothing left to teach it; one it always scores badly on is out of reach. Both give a flat, uninformative reward. What moves the model is the task whose score swings from attempt to attempt.

Have the Solver answer the same task $M$ times, take the standard deviation of the scores, and keep the top $K$ tasks by that number. High scatter means "sometimes it works, sometimes it doesn't" — exactly the frontier of the Solver's ability. The paper grounds this in Bae et al. (2026): the expected policy improvement from training on a task is lower-bounded by the variance of its rewards, so high-variance tasks are the ones with the most room to learn. R-Zero, with a binary verifier, filtered by intermediate accuracy; because the Judge returns continuous scores, dispersion is the continuous analogue of that filter.

One practical caveat: scatter also depends on sampling temperature (the paper uses temperature 1.0 and top-$p$ 0.99, Table 4). Turn the temperature down too far and every rollout looks the same, which quietly disables the selection criterion itself.