4 ms·
Deep learning is the opposite of incremental. For a long time it was not clear whether/how we can learn multi layer networks efficiently. ImageNet changed every
by xiphias 10y ago
Deep learning is the opposite of incremental. For a long time it was not clear whether/how we can learn multi layer networks efficiently. ImageNet changed everything.
- bonoboTP 10y agoMachine learning people basically agree that there weren't any big breakthroughs in deep learning. The success and the hype is mostly a combination of more computing power and more data. The algorithms (convolutional neural network etc.) were invented back in the 1980s and even earlier. There have been some improvements but they are incremental indeed. More use of ReLU, dropout etc. But it's not a new paradigm at all.
- ebalit 10y agoThey weren't any recent breakthroughts. But LSTM and ConvNet are breakthroughts. It just took a lot of times to prove it.
- bonoboTP 10y agoConvnets follow pretty naturally from multilayer perceptrons. Perhaps backpropagation was a breakthrough, enabling the training of ANNs on data, instead of hand-tuning. But the idea of neural nets is very old, going back to Rosenblatt and connectionism.