3 ms·

What is the hypothesis? How was the hypothesis tested?

disgruntledphd2 12y ago
From my reading of the first twenty or so pages, it appears to be a theory of how neural networks model images (the running example). The authors claim that popular neural network architectures can be reduced to particular cases of their model. If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking. Nonetheless, it does not appear to be an experimental paper, rather providing a mathematical theory of some particular classification problems.
- Nvn 12y ago
  > If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking This paper [0] does a pretty good job of giving an explanation for that. [0] http://arxiv.org/abs/1412.6572 http://arxiv.org/abs/1412.6572
  - disgruntledphd2 12y ago
    Honestly, that abstract makes me more upset. If these are due to NN's nature as linear classifers, then we are all in trouble, given that almost everything useful is based off linear models. Given the title of the paper though, I should probably be more hopeful :)
- darkmighty 12y ago
  The adversarial noise issue is not so hard to understand, it's just costly to correct. That link provides the explanation: if your classifier is not very regularized, then the classification regions are going to be close and irregular, s.t. a small vector may lead you from one to another. It's more of a geometrical fact f you think of classification regions in those spaces (of high dimension). Guaranteeing a large minimum distance is hard (essentially why error correcting codes are pretty hard to encode/decode)
- throwaway_bob 12y ago
  the fact that random noise can be classified as strongly belonging to some class, and the fact that classification results can be unstable, is simply a result of the fact the input space is very high dimensional, and the output space is very small (say a few isolated points). That is, if you are discriminatively training a mapping from images in R^(224 x 224 x 3) to 1000 points (class labels), there is going to be a tremendous amount of instability in the inverse direction.

From my reading of the first twenty or so pages, it appears to be a theory of how neural networks model images (the running example). The authors claim that popular neural network architectures can be reduced to particular cases of their model. If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking. Nonetheless, it does not appear to be an experimental paper, rather providing a mathematical theory of some particular classification problems.

Nvn 12y ago
> If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking This paper [0] does a pretty good job of giving an explanation for that. [0] http://arxiv.org/abs/1412.6572 http://arxiv.org/abs/1412.6572
- disgruntledphd2 12y ago
  Honestly, that abstract makes me more upset. If these are due to NN's nature as linear classifers, then we are all in trouble, given that almost everything useful is based off linear models. Given the title of the paper though, I should probably be more hopeful :)
darkmighty 12y ago
The adversarial noise issue is not so hard to understand, it's just costly to correct. That link provides the explanation: if your classifier is not very regularized, then the classification regions are going to be close and irregular, s.t. a small vector may lead you from one to another. It's more of a geometrical fact f you think of classification regions in those spaces (of high dimension). Guaranteeing a large minimum distance is hard (essentially why error correcting codes are pretty hard to encode/decode)
throwaway_bob 12y ago
the fact that random noise can be classified as strongly belonging to some class, and the fact that classification results can be unstable, is simply a result of the fact the input space is very high dimensional, and the output space is very small (say a few isolated points). That is, if you are discriminatively training a mapping from images in R^(224 x 224 x 3) to 1000 points (class labels), there is going to be a tremendous amount of instability in the inverse direction.