teacher

Definition

In distillation, the larger or stronger model whose outputs supply the training signal. The student is fitted to the teacher's predicted distribution — or to its intermediate representations — to retain accuracy at lower inference cost. The word always implies a paired student, and it refers to a model rather than a human annotator; human-written labels are called supervision.

Explainers using this term

• Paper Explained: Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-ImprovementDoes On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement
• Paper explained: Self-OPD — an image generator that distills itself, with no teacherSelf-OPD: On-Policy Distillation for Flow Matching Models without Teacher