feedback

Definition

Signal about the quality of an output, fed back to shape further training or generation. It is the HF in RLHF, where human preference comparisons are converted into a reward used to align a model's behaviour. Increasingly the judge is another model rather than a person — cheaper and faster, but it risks compounding whatever biases the judging model already has.

Explainers using this term

• Alignment, Explained — From RLHF to Constitutional AITraining language models to follow instructions with human feedback
• Recommenders and Embeddings — Same Math as RAG, Different GoalBPR: Bayesian Personalized Ranking from Implicit Feedback
• KL Divergence From Scratch — Measuring the Gap Between Two DistributionsTraining language models to follow instructions with human feedback