Paper Explained: Compile by Training — Turning a Natural-Language Spec into a Function That Runs Locally

"Sort my email into urgent and later" — a spec that vague, turned into a small function you can run locally after about a minute of training. A ground-up walkthrough of Compile by Training: analogy, mechanism, equations, measured numbers.

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

The option between "rules" and "call the chef every time"

"Route an email saying signature needed by EOD to immediate, and a newsletter to wait." You can state the job in one sentence. Try to write it out in if statements and it falls apart immediately — phrasings, politeness levels and personal quirks are endless.

Calling a large remote model for every incoming message is no more comfortable. Every call costs money, adds a network round trip, and ties your business to a provider. Most of the small text jobs a product repeats all day fall into exactly this gap: too fuzzy for conventional code, too narrow and too frequent to justify a big-model call each time.

The paper this article covers is "Compile by Training: Turning Natural-Language Specifications into Local Neural Functions" (Yuntian Deng, Pengyu Nie, Stuart Shieber; EMNLP 2026 System Demonstrations).

Here is the abstract in plain terms. Recurring text functions are easy to describe but hard to implement with rules, while calling a large remote model per input repeatedly incurs cost, latency and provider dependence. The authors propose compile by training: at compile time, teacher models generate task-specific examples, and those examples train a small adapter for a compact interpreter. The resulting function runs without the teachers and can be stored, versioned and composed like ordinary software. On FuzzyBench-Hard — the subset on which the PAW fast compiler produced no exact matches — it reaches 83.6% semantic accuracy, at the price of a higher compile-time cost: roughly a minute rather than the fast compiler's seconds. The authors deploy the compiler as a public interactive service and demonstrate compiled functions in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English–Claudish translator.

The reframing is the point: use large models as build-time tools rather than run-time dependencies (§1). Instead of summoning a top chef for every order, you bring them in once and walk away with a small appliance that reproduces the dish.

Splitting Compile from Run (§2)

The interface the paper defines is strikingly plain.

Here $s$ is the function specification written in natural language (literally the sentence "classify emails as immediate or wait"), $p_{s}$ is the compiled program, $x$ is a new input arriving at run time, and $\hat{y}$ is its output.

The line in words: reading the spec happens once; handling inputs happens forever. Compile a C source file into an executable and you can run that executable a million times without ever starting the compiler again — equation (1) claims nothing more than the same two-stage shape, applied to a specification written in English. The detail worth staring at is that $s$ does not appear in $\texttt{Run}$ at all. At run time, the sentence you wrote is no longer needed.

What matters is that $p_{s}$ is not a model. Every program shares one frozen language model as its interpreter, and each program carries only the adapter and prompt that specialize that interpreter for a single function. The downloadable .paw artifact bundles the spec and those parts, so it can be stored, versioned, cached and reused like any ordinary software artifact.

And here is the sentence that matters most operationally: compilation is a hosted process in which the spec is sent to the PAW service and teacher APIs, but local SDK execution does not send future inputs to PAW or the teachers.

From specification to supervision (§3.1)

A spec describes desired behavior but contains not a single labeled example. So the system has teacher models synthesize a task-specific dataset.

$T(s)$ is "a teacher model handed the specification $s$," each $(x_i, y_i)$ is one worked example of "given this input, produce this output," and $n$ is how many of them you asked for.

The same thing in words: hand a teacher the spec and have it write $n$ practice problems with the answers filled in. The $\sim$ ("drawn from") is doing quiet work — a teacher is a sampler, not a deterministic function, so compiling the same spec twice does not give you the same $D_s$. The reproducibility caveat later in this article is already written into that one symbol. All equation (2) does is translate a sentence of specification into a pile of concrete input–output pairs.