2 ms·
I'm surprised that this doesn't mention cross-entropy, the contrastive loss function used by Facebook's wav2vec2 XLS-R pretraining paper and by OpenAI's CLIP.
by fxtentacle 4y ago
I'm surprised that this doesn't mention cross-entropy, the contrastive loss function used by Facebook's wav2vec2 XLS-R pretraining paper and by OpenAI's CLIP.
- canjobear 4y agoContrastive losses arise from using methods like NCE (mentioned in the post) to approximate cross entropy loss when the partition function is intractable.