3 ms·
Are there many-layered deep networks that aren't convolutional neural nets, or are CNNs practically necessary to make deep networks work? Are there specific ext
by AgentME 10y ago
Are there many-layered deep networks that aren't convolutional neural nets, or are CNNs practically necessary to make deep networks work? Are there specific extra techniques not necessary for CNNs that are necessary to make deep non-convolutional networks work well?
- gok 10y agoThere are successful deep networks with pure feed forward non-convolution layers. There are also deep layers of other exotic non-convolutions flavors like LSTMs and GRUs, particularly useful for sequence-to-sequence tasks like machine translation.
- nl 10y agoIn natural language processing tasks you see a lot of non-CNN architectures. These usually are designed to be able to deal with sequential data, so some kind of "memory" is needed. Sometimes you see this combined with a CNN. There has been a few question answering systems that have one or more CNN layers. In don't entirely understand these designs, but presumably the convultional layers are an attempt to understand the different orders of words. There are lots of techniques that people use to try to make deep networks work well. Mostly theses are about making errors backprog better. One of the most successful recent innovations is the ResNet architectures (https://arxiv.org/abs/1512.03385 https://arxiv.org/abs/1512.03385), and the related highway networks.