4 ms·

Approximating CNNs with Bag-of-Local-Features Models [pdf]

80 points by kawera 8y ago

kawera 8y ago
Short video discussing this paper: https://www.youtube.com/watch?v=QpptSohzuDo https://www.youtube.com/watch?v=QpptSohzuDo
- HNLurker2 8y ago
  +1 for the channel. Very high signal to noise ratio channels among: Brady channel collection (numberphile, cphile etc..)
  - paulgrant999 8y ago
    Brady channel collection? can you provide a full list? I like high snr.
MAXPOOL 8y ago
Open Review https://openreview.net/forum?id=SkfMWhAqYQ https://openreview.net/forum?id=SkfMWhAqYQ
Natsu 8y ago
For anyone else who wanted it: CNN = Convolutional neural network I somehow mis-parsed that as the news network the first time and became very confused for a moment.
amelius 8y ago
Is this closer to how real biological brains process images?
- Veedrac 8y ago
  No, I imagine it's fairly simple to determine that humans use nonlocal context in image recognition.
Veedrac 8y ago
I consider the paper “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness” essentially mandatory follow-up for anyone interested in this. It goes further with its conclusions and introduces a potential solution. By introducing the ‘Stylized-ImageNet’ dataset, they found that they can force the network to learn shape data, improving scores even on standard ImageNet and preventing this BagNet trick from working. https://openreview.net/forum?id=Bygh9j09KX https://openreview.net/forum?id=Bygh9j09KX
- SubiculumCode 8y ago
  seems like texture training would be occur after shape training.
olliej 8y ago
I read this title as being the news network first which was much more interesting :)
jph00 8y ago
The results on this paper are 4x worse than the state of the art of a couple of years ago. Personally I think the paper oversells their results. Creating a much worse model by making it much simpler doesn't seem that interesting, to me...
- sanxiyn 8y ago
  Reading OpenReview, others seem to find the result interesting though.
- albertzeyer 8y ago
  The point is not about creating a new good model and getting good results, but about understanding how the CNN works. And the results of this paper yield some very interesting findings in this respect. Maybe see this two-minutes-paper video for a short explanation: https://www.youtube.com/watch?v=QpptSohzuDo https://www.youtube.com/watch?v=QpptSohzuDo
dgregd 8y ago
It seems that texture detection is for NN the simplest and easiest way to recognize objects. I guess early animal vision system used to work this way. Then predators and prey developed camouflage and mimicry, to fool texture detection NNs. So there was strong evolutionary pressure to recognize shapes, which is much harder task, but essential for survival. Probably that's why people see in the picture a cat (predator) and ConvNet recognizes most obvious thing (elephant) in terms of statistics.
- p1esk 8y ago
  I wonder if capsules are better at recognizing shapes

80 points by kawera 8y ago

Short video discussing this paper: https://www.youtube.com/watch?v=QpptSohzuDo https://www.youtube.com/watch?v=QpptSohzuDo

HNLurker2 8y ago
+1 for the channel. Very high signal to noise ratio channels among: Brady channel collection (numberphile, cphile etc..)
- paulgrant999 8y ago
  Brady channel collection? can you provide a full list? I like high snr.

MAXPOOL 8y ago

Open Review https://openreview.net/forum?id=SkfMWhAqYQ https://openreview.net/forum?id=SkfMWhAqYQ

Natsu 8y ago

For anyone else who wanted it: CNN = Convolutional neural network I somehow mis-parsed that as the news network the first time and became very confused for a moment.

amelius 8y ago

Is this closer to how real biological brains process images?

Veedrac 8y ago
No, I imagine it's fairly simple to determine that humans use nonlocal context in image recognition.

Veedrac 8y ago

I consider the paper “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness” essentially mandatory follow-up for anyone interested in this. It goes further with its conclusions and introduces a potential solution. By introducing the ‘Stylized-ImageNet’ dataset, they found that they can force the network to learn shape data, improving scores even on standard ImageNet and preventing this BagNet trick from working. https://openreview.net/forum?id=Bygh9j09KX https://openreview.net/forum?id=Bygh9j09KX

SubiculumCode 8y ago
seems like texture training would be occur after shape training.

olliej 8y ago

I read this title as being the news network first which was much more interesting :)

jph00 8y ago

The results on this paper are 4x worse than the state of the art of a couple of years ago. Personally I think the paper oversells their results. Creating a much worse model by making it much simpler doesn't seem that interesting, to me...

sanxiyn 8y ago
Reading OpenReview, others seem to find the result interesting though.
albertzeyer 8y ago
The point is not about creating a new good model and getting good results, but about understanding how the CNN works. And the results of this paper yield some very interesting findings in this respect. Maybe see this two-minutes-paper video for a short explanation: https://www.youtube.com/watch?v=QpptSohzuDo https://www.youtube.com/watch?v=QpptSohzuDo

dgregd 8y ago

It seems that texture detection is for NN the simplest and easiest way to recognize objects. I guess early animal vision system used to work this way. Then predators and prey developed camouflage and mimicry, to fool texture detection NNs. So there was strong evolutionary pressure to recognize shapes, which is much harder task, but essential for survival. Probably that's why people see in the picture a cat (predator) and ConvNet recognizes most obvious thing (elephant) in terms of statistics.

p1esk 8y ago
I wonder if capsules are better at recognizing shapes