Also interesting in this context: this recent paper [1] analyzed some of the most popular datasets and found numerous label errors in the test sets. For example

Also interesting in this context: this recent paper [1] analyzed some of the most popular datasets and found numerous label errors in the test sets. For example, they estimated at least 6% of the samples in the ImageNet validation set were misclassified, with surprising consequences when comparing model performances using corrected sets. [1] Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks, https://arxiv.org/pdf/2103.14749.pdf https://arxiv.org/pdf/2103.14749.pdf