3 ms·
> Is there much in the way of datasets of things that are recognizable but triggering incorrect something-or-other (what is it exactly? Semantic sensations?) T
by gwern 2y ago
> Is there much in the way of datasets of things that are recognizable but triggering incorrect something-or-other (what is it exactly? Semantic sensations?)
There are some datasets which collect 'hard' examples. One I love for the title alone is "When does dough become a bagel? Analyzing the remaining mistakes on ImageNet" https://arxiv.org/abs/2205.04596#google https://arxiv.org/abs/2205.04596#google , Vasudevan et al 2022. The errors can be pretty interesting: https://arxiv.org/pdf/2205.04596#page=18 https://arxiv.org/pdf/2205.04596#page=18 You can often see why a model might make a mistake and that you would too. (Like that first one of a swing - I would think it's some sort of large "tripod" too because I have no idea if I've ever seen a swing like that.)
- dekhn 2y agoThe very idea that some objects have a single, correct label is fundamentally a simplification.
- no_news_is 2y agoIn that example, Ground Truth is a tripod and the Prediction is a swing; that is, you'd be right to think it's a tripod. Of course your statement still stands regardless - the errors are interesting.