3 ms·

The real issue here is that imagenet doesn't have enough funny glasses. Same reason detection networks will think that gravel is broccoli, a network learns just

The real issue here is that imagenet doesn't have enough funny glasses. Same reason detection networks will think that gravel is broccoli, a network learns just enough to distinguish between the categories it's presented. Improvements in datasets, transfer learning, and online learning will help. Unfortunately this underscores the issue that giants such as google have more pictures of funny glasses than anyone else...

Jweb_Guru 10y ago
No, that's not the only issue (it's not just that it doesn't have enough adversarial samples), as you would know if you'd read the papers involved.
- paulsutter 10y ago
  The central issue is that networks are trained on a limited database of images (usually imagenet) and then frozen in time. Going beyond this involves evolving the nature of networks, and there's nothing the human brain can do that can't be done. The design of networks is limited by the dataset, since researchers are just trying to hit the metric for the dataset. Better datasets will lead to better networks. Better data eventually means billions of cameras, for example.
  - kem 10y ago
    The issue is that the networks are using a trivial, nonessential stimulus feature to base classification decisions on. It's like basing the decision on a very small-eigenvalue feature eigenvector. That could be a training set issue, or it could be something about the network structure. The article was kind of interesting to me because it reveals that networks are probably sometimes making decisions on highly discriminating but non-essential stimulus features. It's like the networks might have a high success rate with large number of replicated examples, over sample size, but not over a large number of distinct examples. They're overfitting, but in a way that isn't immediately obvious because the test stimuli tend to be limited. My guess is the answer is to incorporate into the training set a lot of quasi-random or structured but abstract images as controls for training.
    - paulsutter 10y ago
      Yes exactly, they just barely work for the examples shown them. It is caused by the training set, the network structure, and the metric which are all intertwined. The design of the network is a direct consequence of the dataset+metric because researchers focus on accuracy scores against the test data. Given a better dataset and metric, researchers will solve it with a better network design. I guarantee it.
zeteo 10y ago
> The real issue here is that imagenet doesn't have enough funny glasses. But we don't actually know the real issue, do we? You can't easily debug a neural network. Sure, you can throw ten times more data at it and hope that fixes it. But the more data you throw in, the less you understand what's happening - which affects your ability to anticipate the next problem.

Jweb_Guru 10y ago

No, that's not the only issue (it's not just that it doesn't have enough adversarial samples), as you would know if you'd read the papers involved.

paulsutter 10y ago
The central issue is that networks are trained on a limited database of images (usually imagenet) and then frozen in time. Going beyond this involves evolving the nature of networks, and there's nothing the human brain can do that can't be done. The design of networks is limited by the dataset, since researchers are just trying to hit the metric for the dataset. Better datasets will lead to better networks. Better data eventually means billions of cameras, for example.
- kem 10y ago
  The issue is that the networks are using a trivial, nonessential stimulus feature to base classification decisions on. It's like basing the decision on a very small-eigenvalue feature eigenvector. That could be a training set issue, or it could be something about the network structure. The article was kind of interesting to me because it reveals that networks are probably sometimes making decisions on highly discriminating but non-essential stimulus features. It's like the networks might have a high success rate with large number of replicated examples, over sample size, but not over a large number of distinct examples. They're overfitting, but in a way that isn't immediately obvious because the test stimuli tend to be limited. My guess is the answer is to incorporate into the training set a lot of quasi-random or structured but abstract images as controls for training.
  - paulsutter 10y ago
    Yes exactly, they just barely work for the examples shown them. It is caused by the training set, the network structure, and the metric which are all intertwined. The design of the network is a direct consequence of the dataset+metric because researchers focus on accuracy scores against the test data. Given a better dataset and metric, researchers will solve it with a better network design. I guarantee it.

zeteo 10y ago

> The real issue here is that imagenet doesn't have enough funny glasses. But we don't actually know the real issue, do we? You can't easily debug a neural network. Sure, you can throw ten times more data at it and hope that fixes it. But the more data you throw in, the less you understand what's happening - which affects your ability to anticipate the next problem.