3 ms·

I believe that the kernels learned by a deep net (especially the detailed ones) are basically what this guy is talking about (a small nnet that recognizes basic

I believe that the kernels learned by a deep net (especially the detailed ones) are basically what this guy is talking about (a small nnet that recognizes basically one feature). I suppose you could sample a large number of capsules, but that would be equivalent to just making a bigger deep net.

malmsteen 9y ago
It's probably more than that otherwise that guy wouldn't waste his time.
- ccozan 9y ago
  yes, is about feeding the 3D context too. It means to recognize a feature once, and then give a spatial translation, is able to say, yes is the same feature, just turned 30deg right, 40deg up, for example, without having to train the model with a _picture_ of an object taken from all sides and perspectives. Humans use binocular vision [1], but AI can be programmed to do more. This is practically introducing AI to the real world: an object is more than the picture of it. [1] https://en.wikipedia.org/wiki/Binocular_vision https://en.wikipedia.org/wiki/Binocular_vision
visarga 9y ago
There is dynamic routing between capsules, compared to static routing in normal deep nets. The routing itself is learned.
Nokinside 9y ago
Small correction: "this guy" is "the guy" behind deep learning revolution.
- carapace 9y ago
  >Abstract. In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. http://people.idsia.ch/~juergen/deep-learning-overview.html http://people.idsia.ch/~juergen/deep-learning-overview.html

malmsteen 9y ago

It's probably more than that otherwise that guy wouldn't waste his time.

ccozan 9y ago
yes, is about feeding the 3D context too. It means to recognize a feature once, and then give a spatial translation, is able to say, yes is the same feature, just turned 30deg right, 40deg up, for example, without having to train the model with a _picture_ of an object taken from all sides and perspectives. Humans use binocular vision [1], but AI can be programmed to do more. This is practically introducing AI to the real world: an object is more than the picture of it. [1] https://en.wikipedia.org/wiki/Binocular_vision https://en.wikipedia.org/wiki/Binocular_vision

visarga 9y ago

There is dynamic routing between capsules, compared to static routing in normal deep nets. The routing itself is learned.

Nokinside 9y ago

Small correction: "this guy" is "the guy" behind deep learning revolution.

carapace 9y ago
>Abstract. In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. http://people.idsia.ch/~juergen/deep-learning-overview.html http://people.idsia.ch/~juergen/deep-learning-overview.html