5 ms·

No, that's not the only issue (it's not just that it doesn't have enough adversarial samples), as you would know if you'd read the papers involved.

paulsutter 10y ago
The central issue is that networks are trained on a limited database of images (usually imagenet) and then frozen in time. Going beyond this involves evolving the nature of networks, and there's nothing the human brain can do that can't be done. The design of networks is limited by the dataset, since researchers are just trying to hit the metric for the dataset. Better datasets will lead to better networks. Better data eventually means billions of cameras, for example.
- kem 10y ago
  The issue is that the networks are using a trivial, nonessential stimulus feature to base classification decisions on. It's like basing the decision on a very small-eigenvalue feature eigenvector. That could be a training set issue, or it could be something about the network structure. The article was kind of interesting to me because it reveals that networks are probably sometimes making decisions on highly discriminating but non-essential stimulus features. It's like the networks might have a high success rate with large number of replicated examples, over sample size, but not over a large number of distinct examples. They're overfitting, but in a way that isn't immediately obvious because the test stimuli tend to be limited. My guess is the answer is to incorporate into the training set a lot of quasi-random or structured but abstract images as controls for training.
  - paulsutter 10y ago
    Yes exactly, they just barely work for the examples shown them. It is caused by the training set, the network structure, and the metric which are all intertwined. The design of the network is a direct consequence of the dataset+metric because researchers focus on accuracy scores against the test data. Given a better dataset and metric, researchers will solve it with a better network design. I guarantee it.

paulsutter 10y ago

The central issue is that networks are trained on a limited database of images (usually imagenet) and then frozen in time. Going beyond this involves evolving the nature of networks, and there's nothing the human brain can do that can't be done. The design of networks is limited by the dataset, since researchers are just trying to hit the metric for the dataset. Better datasets will lead to better networks. Better data eventually means billions of cameras, for example.

kem 10y ago
The issue is that the networks are using a trivial, nonessential stimulus feature to base classification decisions on. It's like basing the decision on a very small-eigenvalue feature eigenvector. That could be a training set issue, or it could be something about the network structure. The article was kind of interesting to me because it reveals that networks are probably sometimes making decisions on highly discriminating but non-essential stimulus features. It's like the networks might have a high success rate with large number of replicated examples, over sample size, but not over a large number of distinct examples. They're overfitting, but in a way that isn't immediately obvious because the test stimuli tend to be limited. My guess is the answer is to incorporate into the training set a lot of quasi-random or structured but abstract images as controls for training.
- paulsutter 10y ago
  Yes exactly, they just barely work for the examples shown them. It is caused by the training set, the network structure, and the metric which are all intertwined. The design of the network is a direct consequence of the dataset+metric because researchers focus on accuracy scores against the test data. Given a better dataset and metric, researchers will solve it with a better network design. I guarantee it.