> Somebody (Geoffrey Hinton, Andrew Ng, I don't remember exactly) made the point that neural networks were in a pretty moribund place, until the confluence of t

> Somebody (Geoffrey Hinton, Andrew Ng, I don't remember exactly) made the point that neural networks were in a pretty moribund place, until the confluence of three things came along: more compute power, more data, and algorithmic improvements. Given that, one wonders what other older ideas are waiting to be resurrected by a similar confluence. Andrew Ng says something like that on his deep learning Coursera course. Also, Animats mentions perceptrons. A neural network is a generalized (combination of) perceptron(s) on steroids. That ceiling was basically removed again with the advent of the above-mentioned improvements. Similarly, the gradient descent used to optimize a neural network resembles hill climbing, as well.