supervision

Definition

The training signal that tells a model what the right answer is. Methods are largely classified by where it comes from: human labels (supervised), the data's own structure such as predicting masked or next tokens (self-supervised), noisy or partial signals (weak supervision), or none (unsupervised). Since labels are the expensive part, much of the field's progress came from extracting more signal from unlabeled data.

Explainers using this term

• On-Policy Distillation — Learning From What the Student Actually WritesOn-Policy Self-Distillation without Any Supervision
• Representing Sound — Mel Spectrograms and Audio TokensRobust Speech Recognition via Large-Scale Weak Supervision
• How VLMs Came Together — Wiring a Vision Encoder into an LLMLearning Transferable Visual Models From Natural Language Supervision