5 ms·

Hinton explains the concept of capsules in this video: https://www.youtube.com/watch?v=rTawFwUvnLE https://www.youtube.com/watch?v=rTawFwUvnLE Which is a lot b

Hinton explains the concept of capsules in this video: https://www.youtube.com/watch?v=rTawFwUvnLE https://www.youtube.com/watch?v=rTawFwUvnLE Which is a lot better than reading someone tell you about this new idea that is called "capsules" but doesn't go into detail. The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net). Better source, with date: http://techtv.mit.edu/collections/bcs/videos/30698-what-s-wrong-with-convolutional-nets http://techtv.mit.edu/collections/bcs/videos/30698-what-s-wr... (December 2014, for the lazy).

olewhalehunter 9y ago
at a first glance this looks like it's going back to old 80s research on neural representations, back then this kind of stuff was known as Parallel Distributed Processing https://mitpress.mit.edu/books/parallel-distributed-processing https://mitpress.mit.edu/books/parallel-distributed-processi...
- grandalf 9y ago
  Indeed. A blast from the past. Oddly the capsule approach is how I naively thought image recognition worked until I learned more about it.
deleted 9y ago
[deleted]
pmikesell 9y ago
Thank you for the link to the video - this was excellent. His key critique of convnets is that (paraphrasing) all of these pooling layer (max, avg, whatever) are really a very poor way of attempting to solve what is actually a routing problem.
thomasahle 9y ago
Apparently, one of the main changes since the talk, is that then he used a "probabilistic unit" next to his "property vector". Now the length of the property vector indicates the probability.
- mannigfaltig 9y ago
  In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb https://openreview.net/forum?id=HJWLfGWRb
SophosQ 9y ago
The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net). The paper however details their experiments on Cifar-10 and other datasets in the discussion section. They don't produce good enough results but the paper proposes certain hypotheses for the poorer performance and that it could be overcome in the future. Arxiv-link: https://arxiv.org/abs/1710.09829 https://arxiv.org/abs/1710.09829

olewhalehunter 9y ago

at a first glance this looks like it's going back to old 80s research on neural representations, back then this kind of stuff was known as Parallel Distributed Processing https://mitpress.mit.edu/books/parallel-distributed-processing https://mitpress.mit.edu/books/parallel-distributed-processi...

grandalf 9y ago
Indeed. A blast from the past. Oddly the capsule approach is how I naively thought image recognition worked until I learned more about it.

deleted 9y ago

[deleted]

pmikesell 9y ago

Thank you for the link to the video - this was excellent. His key critique of convnets is that (paraphrasing) all of these pooling layer (max, avg, whatever) are really a very poor way of attempting to solve what is actually a routing problem.

thomasahle 9y ago

Apparently, one of the main changes since the talk, is that then he used a "probabilistic unit" next to his "property vector". Now the length of the property vector indicates the probability.

mannigfaltig 9y ago
In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb https://openreview.net/forum?id=HJWLfGWRb

SophosQ 9y ago

The only thing is that, when this presentation was given, it seems they hadn't worked much more than MNIST (so the new thing now would be the toys-recognition net). The paper however details their experiments on Cifar-10 and other datasets in the discussion section. They don't produce good enough results but the paper proposes certain hypotheses for the poorer performance and that it could be overcome in the future. Arxiv-link: https://arxiv.org/abs/1710.09829 https://arxiv.org/abs/1710.09829