3 ms·
I strongly suspect the inability to do negation comes from the fact that the training dataset contains virtually no images tagged with negated descriptors. It
by bloaf 4y ago
I strongly suspect the inability to do negation comes from the fact that the training dataset contains virtually no images tagged with negated descriptors. It wouldn't surprise me to see it correctly handle negations that might be more common in image tags (e.g. "a car not running" might be a sufficiently common phrase for "broken down car").
- gwern 4y agoSeems straightforward to check something like LAION-400M's captions to see how often negations show up. If it's simply a lack of data, that should be easy to fix. Given an image caption, every other caption is a relevant negation: 'a photo of a happy dog' is also not 'a photo of a sad cat', and can be turned into the caption 'a photo of a happy dog and not a photo of a sad cat'. So lots of obvious data augmentation and synthetic data approaches to apply there to boost negation.