preferences

Definition

Judgments about which of two candidate outputs is better. Pairwise comparisons are used in place of absolute scores because annotators are far more consistent comparing than rating, and both RLHF and DPO consume them directly as the training signal. Whose preferences you collect is a design decision rather than a detail: the annotator pool and the instructions given determine what the aligned model comes to treat as good.