3 ms·

Our datasets are flawed. ImageNet has an error rate of ~5.8%

• V__ 5y ago
  When I first started playing around with neural networks, I noticed that even the MNIST numbers had some images mislabeled. I kinda suspect that most people never really check imported datasets for errors.
• cinntaile 5y ago
  Another somewhat related "issue" is that the mean and std used in ImageNet has unknown origins. It's based off the data but it's not clear exactly how they ended up with those numbers. [0] [0] https://github.com/pytorch/vision/issues/1439 https://github.com/pytorch/vision/issues/1439
  • belval 5y ago
    Interestingly, these are also used for a lot of CV models that have nothing to do with ImageNet.
    • cinntaile 5y ago
      I wonder if that's because the numbers are general enough across many different tasks or if normalization just isn't as important as we assume?