3 ms·
It's true that the beginning of the talk is about how people abandoned backprop trained neural networks because they often underperformed SVMs, but the rest of
by jbarrow 12y ago
It's true that the beginning of the talk is about how people abandoned backprop trained neural networks because they often underperformed SVMs, but the rest of the talk is about deep learning, which introduced a new generation of neural nets (since 2006) that are the current state of the art for a lot of problems.
In fact, deep neural networks are trained in an unsupervised manner at first, but then back propagation is used to "fine tune" and improve the results. Because they require unlabeled data sets, and can perform so well, research into neural networks has experienced a recent resurgence.
By the way, any talk by Geoff Hinton is fantastic. If you are interested in neural networks and their capabilities, and you haven't already seen it, his Coursera course [1] builds from a simple linear perceptron to the current deep learning methods.
[1] https://class.coursera.org/neuralnets-2012-001 https://class.coursera.org/neuralnets-2012-001 (You'll have to sign in to see it)
- agibsonccc 12y agoI want to add to this, that there's a lot of work that doesn't require pretraining. I recently implemented the more advanced hessian free optimization methods that don't require pretraining in my deep learning framework. The results are amazing. I'm hoping to demonstrate a lot of the tradeoffs of the different methods in a more comprehensive manner here shortly. This was an updated extension by some of hinton's students. The paper I implemented was: http://www.cs.toronto.edu/~rkiros/papers/shf13.pdf http://www.cs.toronto.edu/~rkiros/papers/shf13.pdf
- taylorbuley 12y agoThanks for the extra resources. Any more are very welcome. Neural nets sometimes feel like quite the black box, despite their ease of implementation and apparent power.