3 ms·
TL;DR: The authors pretrain the model to classify images into Wordnet synsets[a] that appear in the caption, using a standard Cross Entropy loss. They keep the
by cs702 2y ago
TL;DR: The authors pretrain the model to classify images into Wordnet synsets[a] that appear in the caption, using a standard Cross Entropy loss. They keep the number of classes relatively small by removing any synsets that don't show up in captions at least 500 times in the dataset. It seems to work well.
My immediate question is: Why not classify among the entire hierarchy of all Wordnet synsets?
---
[a] https://wordnet.princeton.edu/ https://wordnet.princeton.edu/
- cs702 2y agoI've tried this for Wordnet hierarchical classification with a Standard Cross Entropy loss: https://github.com/glassroom/heinsen_tree#sample-usage-with-wordnet https://github.com/glassroom/heinsen_tree#sample-usage-with-... It worked for me, but I had to modify the code to use all hypernym paths, giving me 147,200 classes, one per path. English only. For synsets with more than one path, I split target probability mass over their paths. For prediction, I added the predicted probs of hypernym paths ending at each synset.