3 ms·
So it’s been almost two years since the capsules paper, and they still have nothing but mnist results? A bit underwhelming...
by p1esk 7y ago
So it’s been almost two years since the capsules paper, and they still have nothing but mnist results? A bit underwhelming...
- tw1010 7y agoHas anything actually happened in the last two years? Genuinely curious.
- p1esk 7y agoAs a comparison, look at the transformer paper (also 2017), and the progress it has triggered.
- ivalm 7y agoI think there are two big things: 1. Attention mechanism (transformers). Really moved forward NLP. It allows long distance dependencies without the need for sequential processing (like in RNNs). Generally scales much better (see BERT/GPT-2/XL-Net/etc) 2. Separable convolutions. Significantly sped-up the standard CV networks. (mobile net, but now everything, really).
- SubiculumCode 7y agohttps://arxiv.org/abs/1906.06818 https://arxiv.org/abs/1906.06818 says June 2019. To which paper do you refer?
- muizelaar 7y ago"Matrix capsules with EM routing"
- MAXPOOL 7y agoCapsule networks are attempt to get past the limits of current approach to deep learning. It took 20 years until the the current paradigm in deep learning started to generate results. You start with toy problems and explore. Hinton's "What is wrong with convolution neural nets?" is still true to the state of the art in convnets. Gradually turning features into objects like we do in deep learning is limiting. Distinguishing feature integration from object detection like humans do and how to do it is an open question. It can take another 10-15 years.
- p1esk 7y agoOr maybe it’s just another overcomplicated model that’s ultimately a dead end, like, say, RBMs, which were also pushed hard by Hinton (before DL breakthrough in 2012).