4 ms·

Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane. So I'm a bit skeptical that this tells

Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane. So I'm a bit skeptical that this tells us something about the difference between cnns and humans. Especially since the models did seem to adapt well to the modified domain when trained on it.

hackinthebochs 9y ago
>Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane. I don't see how this follows. What we're not focused on is blurred, yes, but what we're focused on is at least partially related to our ability to recognize patterns (e.g. reading from peripheral vision is very difficult). The fact that CNNs are tricked by imperceptible perturbations while the semantic content is held constant is highly informative information, don't reject it for superficial reasons. The standard fair of CNN+RELU+Residuals are very powerful modelling tools but that also means they're prone to model degenerate regularities if they exist. This paper shows that they do exist and that these models are picking up on them at least to some significant extent.
- moultano 9y ago
  >I don't see how this follows. What we're not focused on is blurred, yes, but what we're focused on is at least partially related to our ability to recognize patterns (e.g. reading from peripheral vision is very difficult). Object detection from peripheral vision is not difficult though. I think you're overestimating how much of our vision is actually clear. >The fact that CNNs are tricked by imperceptible perturbations while the semantic content is held constant is highly informative information, don't reject it for superficial reasons. Yes, but that is not this paper. These perturbations are not imperceptible, and unlike other adversarial examples the model adapted well when allowed to train on them. Also, it looks like the model did reasonable well on the random filtered versions, only failing on the blurred versions. The random filtered images looked much more corrupted to me than the blurred ones, which is consistent with blurred images being part of the training for the human visual system, but not randomly filtered ones.
  - yorwba 9y ago
    The radial masking in the Fourier domain is pretty much imperceptible for me, but it seems to cause the most problems for the CNNs.
    - moultano 9y ago
      I'm pretty sure that's because you are constantly trained on that type of image in your day to day life. "Radial masking in the Fourier domain" is pretty much identical to an image that it is out of focus or seen with a low resolution part of your eye. (Most of your vision.)
  - hackinthebochs 9y ago
    >Object detection from peripheral vision is not difficult though. Because we understand high level semantic information, which is largely robust against blurring. >The random filtered images looked much more corrupted to me than the blurred ones, which is consistent with blurred images being part of the training for the human visual system, It's also consistent with the neural network training on surface level regularities, which a random filter would corrupt at a much lower rate than a blur filter would.
yorwba 9y ago
> Especially since the models did seem to adapt well to the modified domain when trained on it. But only when trained on it. The models trained on only one modified domain didn't generalize well to the others. Although the model trained on all datasets together generalized quite well to the corresponding test sets, that doesn't mean that there isn't some other kind of modification that can fool it. This is what the authors say: Our last set of experiments involves training on the fully augmented training set, which now enjoys a variance of its Fourier image statistics. We note that this sort of data augmentation was able to close the generalization gap. However, we stress that it is doubtful that this sort of data augmentation scheme is sufficient to enable a machine learning model to truly learn the semantic concepts present in a dataset. Rather this sort of data augmentation scheme is analogous to adversarial training [40, 9]: there is a nontrivial regularization benefit, but it is not a solution to the underlying problem of not learning high level semantic concepts, nor do we aim to present it as such.
- wiz21c 9y ago
  Is it me or the part of the article you cite basically says : CNN are nice statistical tools but are nowhere close to forming practical conceptual understanding of the data set they're trained on ?
  - mannykannot 9y ago
    The first part of this paragraph (up to 'however') seems to be saying that training with their modified images did improve the ability of the resulting networks to generalize, while the remainder of the paragraph seems to cautioning against assuming that this means the problem is solved.
mannykannot 9y ago
While the two clauses in your first sentence both seem to state established facts, is there any evidence that the first is explained by the second?
- moultano 9y ago
  If we couldn't deal with blurry images through some means, we couldn't recognize objects because our fovea is too small to contain them (a quarter held at arm's length.)
  - mannykannot 9y ago
    This is an unrealistic model of vision - we don't perceive the world as a tiny spot of clarity surrounded by a blur. We scan to compensate. To say that we compensate for the blurring is quite different than saying we can do better (than CNNs?) with blurred images because most of the visual field is blurred. Did you mean to say that we are good at visual recognition despite most of the visual field being blurred? If so, I do not think it throws much light on what is happening in CNNs.
    - moultano 9y ago
      Compensation happens within our visual system as well, not just by saccading. As an experiment, with your eyes focused on your phone, identify the objects around you without moving your eyes. You can still do it very well, and likely need to do it all the time.
      - mannykannot 9y ago
        Yes, we are agreed that compensation happens, but how does that justify "Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane"? Was that statement supposed to begin "we know that..."? - because, as it stands, the sentence states that our generally blurry visual field causally explains why we can identify things in blurred images, something that I still have seen no evidence for.
jasonmaydie 9y ago
I think the real question that needs to be asked is.. does predicting things mean the same thing as learning? CNNs can predict fairly well but do they actually learn? This paper says no.
- moultano 9y ago
  This paper says that CNNs learns from different features than we do and that can be filtered out in the frequency domain. That's still learning.

hackinthebochs 9y ago

>Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane. I don't see how this follows. What we're not focused on is blurred, yes, but what we're focused on is at least partially related to our ability to recognize patterns (e.g. reading from peripheral vision is very difficult). The fact that CNNs are tricked by imperceptible perturbations while the semantic content is held constant is highly informative information, don't reject it for superficial reasons. The standard fair of CNN+RELU+Residuals are very powerful modelling tools but that also means they're prone to model degenerate regularities if they exist. This paper shows that they do exist and that these models are picking up on them at least to some significant extent.

moultano 9y ago
>I don't see how this follows. What we're not focused on is blurred, yes, but what we're focused on is at least partially related to our ability to recognize patterns (e.g. reading from peripheral vision is very difficult). Object detection from peripheral vision is not difficult though. I think you're overestimating how much of our vision is actually clear. >The fact that CNNs are tricked by imperceptible perturbations while the semantic content is held constant is highly informative information, don't reject it for superficial reasons. Yes, but that is not this paper. These perturbations are not imperceptible, and unlike other adversarial examples the model adapted well when allowed to train on them. Also, it looks like the model did reasonable well on the random filtered versions, only failing on the blurred versions. The random filtered images looked much more corrupted to me than the blurred ones, which is consistent with blurred images being part of the training for the human visual system, but not randomly filtered ones.
- yorwba 9y ago
  The radial masking in the Fourier domain is pretty much imperceptible for me, but it seems to cause the most problems for the CNNs.
  - moultano 9y ago
    I'm pretty sure that's because you are constantly trained on that type of image in your day to day life. "Radial masking in the Fourier domain" is pretty much identical to an image that it is out of focus or seen with a low resolution part of your eye. (Most of your vision.)
- hackinthebochs 9y ago
  >Object detection from peripheral vision is not difficult though. Because we understand high level semantic information, which is largely robust against blurring. >The random filtered images looked much more corrupted to me than the blurred ones, which is consistent with blurred images being part of the training for the human visual system, It's also consistent with the neural network training on surface level regularities, which a random filter would corrupt at a much lower rate than a blur filter would.

yorwba 9y ago

> Especially since the models did seem to adapt well to the modified domain when trained on it. But only when trained on it. The models trained on only one modified domain didn't generalize well to the others. Although the model trained on all datasets together generalized quite well to the corresponding test sets, that doesn't mean that there isn't some other kind of modification that can fool it. This is what the authors say: Our last set of experiments involves training on the fully augmented training set, which now enjoys a variance of its Fourier image statistics. We note that this sort of data augmentation was able to close the generalization gap. However, we stress that it is doubtful that this sort of data augmentation scheme is sufficient to enable a machine learning model to truly learn the semantic concepts present in a dataset. Rather this sort of data augmentation scheme is analogous to adversarial training [40, 9]: there is a nontrivial regularization benefit, but it is not a solution to the underlying problem of not learning high level semantic concepts, nor do we aim to present it as such.

wiz21c 9y ago
Is it me or the part of the article you cite basically says : CNN are nice statistical tools but are nowhere close to forming practical conceptual understanding of the data set they're trained on ?
- mannykannot 9y ago
  The first part of this paragraph (up to 'however') seems to be saying that training with their modified images did improve the ability of the resulting networks to generalize, while the remainder of the paragraph seems to cautioning against assuming that this means the problem is solved.

mannykannot 9y ago

While the two clauses in your first sentence both seem to state established facts, is there any evidence that the first is explained by the second?

moultano 9y ago
If we couldn't deal with blurry images through some means, we couldn't recognize objects because our fovea is too small to contain them (a quarter held at arm's length.)
- mannykannot 9y ago
  This is an unrealistic model of vision - we don't perceive the world as a tiny spot of clarity surrounded by a blur. We scan to compensate. To say that we compensate for the blurring is quite different than saying we can do better (than CNNs?) with blurred images because most of the visual field is blurred. Did you mean to say that we are good at visual recognition despite most of the visual field being blurred? If so, I do not think it throws much light on what is happening in CNNs.
  - moultano 9y ago
    Compensation happens within our visual system as well, not just by saccading. As an experiment, with your eyes focused on your phone, identify the objects around you without moving your eyes. You can still do it very well, and likely need to do it all the time.
    - mannykannot 9y ago
      Yes, we are agreed that compensation happens, but how does that justify "Humans do well with blurred images because our vision is blurry everywhere except our fovea and our current focal plane"? Was that statement supposed to begin "we know that..."? - because, as it stands, the sentence states that our generally blurry visual field causally explains why we can identify things in blurred images, something that I still have seen no evidence for.

jasonmaydie 9y ago

I think the real question that needs to be asked is.. does predicting things mean the same thing as learning? CNNs can predict fairly well but do they actually learn? This paper says no.

moultano 9y ago
This paper says that CNNs learns from different features than we do and that can be filtered out in the frequency domain. That's still learning.