clip

Definition

OpenAI's contrastive image-text model, trained on large numbers of image-caption pairs to place matching images and texts near each other in a shared embedding space. Because class names can be written as text, it classifies zero-shot: no labeled examples for the target categories are needed. Today CLIP is used less as a standalone classifier than as the vision encoder inside multimodal LLMs and text-to-image systems, and as a similarity metric for evaluating generated images.

In the textbook

• CLIP — Words and Images on One Map