3 ms·
Is the April Fool's joke of this article (dated April 1) the fact that the error rate is 5.8% for ImageNet? Because last I heard, it wasn't exactly breaking ne
by supernova87a 6y ago
Is the April Fool's joke of this article (dated April 1) the fact that the error rate is 5.8% for ImageNet? Because last I heard, it wasn't exactly breaking news that any training dataset has some intrinsic error rate, or that some error rate means that the models are "flawed".
Maybe I would find this more interesting if the question were, "how good does good enough have to be"?
- PeterisP 6y agoIf a good ImageNet model gets 88-90% accuracy on the test set (so, 10%-12% error rate) then if it turns out that 5.8% of the test set "gold standard" labels are actually wrong is a very big deal, it would mean that half of the measured differences aren't actually real but an artifact of the original human labelers' mistakes.
- iudqnolq 6y agoThe interesting point they make is that the error is non-random, and a significt part of why complex models are better than simple models is they're better at guessing the wrong labels.