4 ms·
Whatever happened to shallow learning (or you know the regular learning) everyone did before deep learning. Anyone still doing that? Is this like BigData. As
by rdtsc 12y ago
Whatever happened to shallow learning (or you know the regular learning) everyone did before deep learning.
Anyone still doing that?
Is this like BigData. As soon as someone mentioned BigData, anyone in the world who touched data all of the sudden did BigData.
So is this something coming out of Google and Facebook and such and everyone else in Academia is happily building SVMs and 2 layer neural networks or some new discovery happend and overturned the whole ML and AI field on its head?
> Crucially, the number of units required to solve these problems is far from exponential --- on the contrary, the number of units required is often so “small” that it is even possible, using current hardware,
Number of units is not what's important. There are "only" what, 10B (100B?)neurons in the brain? But isn't the trick in the connections. And there are orders of magnitudes more connectsion (hundreds of trillions). Not exponential but even quadratic at those numbers is still quite large.
- unlikelymordant 12y agodeep learning is only applicable to some problems e.g. speech recognition and image classification where there is mountains of data to train many network parameters but also a certain level of complexity in the features. Most simple problems with a few thousand feature vectors for training are still solved by SVMs etc. Deep learning is getting excellent results in the problems it is good at though, better than any other classifier. Remember SVMs were being thrown around as the ML wunderkind prior to Deep learning. After a while people figure out exactly what some things are good at and some things not so good.
- wodenokoto 12y agoA rather sizable part of the article talk about why large deep network can solve problems that shallow networks can't. I won't say that the ML field got turned on its head by deep networks. I think people from the very start have wanted to try and make networks deeper, if only they'd known how.
- nl 12y agoWhatever happened to shallow learning (or you know the regular learning) everyone did before deep learning. Deep learning happened, and it pretty much always beats other approaches. Saying that sounds unbelievable, so here's a quote from Pete Warden: I know I’m a broken record on deep learning, but almost everywhere it’s being applied it’s doing better than techniques that people have been developing for decades[1] There's a great paper from a group of researchers who set out to prove that their technique, which they had many years of experience in (SVMs?) was just as good as deep learning (I can't remember their field). They ended up proving the opposite, and switched their whole lab over to doing deep learning. I can't find the paper (!!) so I'll refer you to [2] instead. [1] http://petewarden.com/2015/01/01/five-short-links-76/ http://petewarden.com/2015/01/01/five-short-links-76/ [2] http://petewarden.com/2014/06/10/why-is-everyone-so-excited-about-deep-learning/ http://petewarden.com/2014/06/10/why-is-everyone-so-excited-...