contrastive

Definition

Learning representations by pulling matched pairs together in embedding space and pushing unmatched ones apart. Its appeal is that positives and negatives can be constructed without manual labels — two augmentations of the same image, or an image and its caption, as in CLIP. The details that decide whether it works are how negatives are sampled and how many are seen at once, which is why contrastive methods are notoriously sensitive to batch size.

Explainers using this term

• Self-Supervised Learning — The Day Unlabeled Data Became an AssetA Simple Framework for Contrastive Learning of Visual Representations
• Mutual Information — Putting a Number on What You KnowRepresentation Learning with Contrastive Predictive Coding