student

Definition

In knowledge distillation, the smaller model trained to reproduce a larger teacher's behavior. It learns from the teacher's full output distribution — and sometimes intermediate representations — rather than from hard labels alone, which conveys more information per example and often beats training the same small architecture directly on the original data. Student and teacher are roles in a training setup, not architectures.

Explainers using this term

• Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* StudentStudentSim: Training LLM-based Student Simulators