From the paper on the CLIP embedder it appears that they use a form of contrastive loss that maximizes the cosine similarity between related images and prompts,

From the paper on the CLIP embedder it appears that they use a form of contrastive loss that maximizes the cosine similarity between related images and prompts, but also minimizes the same between unrelated prompts & images. See section 2.3 of the CLIP paper: https://arxiv.org/pdf/2103.00020.pdf https://arxiv.org/pdf/2103.00020.pdf Also, the writeup on OpenAI's blog: https://openai.com/blog/clip/ https://openai.com/blog/clip/