distilling

Definition

The gerund of distillation, used when a title names what is being transferred and where it goes: "distilling X into Y." The object is not always a model — papers distill multi-step sampling procedures into few-step ones, search into a policy, or a long prompt into weights. The common thread is compressing an expensive process into a cheaper one that mimics its outputs.

Explainers using this term

• Paper Explained: Repo-To-Skill — Distilling GitHub Repositories Into Skills an AI Can UseRepo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills
• Distillation vs. Quantization vs. Pruning — Three Roads to a Smaller ModelDistilling the Knowledge in a Neural Network
• A Field Guide to Distillation Recipes — logit, feature, attention, selfDistilling the Knowledge in a Neural Network