3 ms·
Omitting the other members of the LBH conspiracy and Schmidhuber is unfortunate, but I agree with the idea that the number one reason deep learning is working n
by pinouchon 10y ago
Omitting the other members of the LBH conspiracy and Schmidhuber is unfortunate, but I agree with the idea that the number one reason deep learning is working now is scale. Hinton also says it himself, for example (at 5m45 in the video below): "What was wrong in the 80 was that we didn't have enough data and we didn't have enough compute power. [...] Those were the main things that were wrong".
He gives a "brief history of backpropagation" here: https://youtu.be/l2dVjADTEDU?t=4m35s https://youtu.be/l2dVjADTEDU?t=4m35s
- lars 10y agoI agree that scale is an important factor in deep learning's success, but that Google experiment ended up being a good example of how not to do it. They used 16000 CPU cores to get that cat detector. A short while later, a group at Baidu were able to replicate the same network with only 3 computers with 4 GPUs each. (The latter group was also lead by Andrew Ng.)
- espadrine 10y agoIncidentally, seeing the speaker set up an overkill neural network for a trivial classification problem seemed off to me. Unsurprisingly, at least 75% of the neurons were unused. Throwing a phenomenal amount of neurons at a problem is not a goal; using a minimal amount to solve it in a given time budget is. The statement at the end of the video, “all the serious applications from here on out need to have deep learning and AI inside”, seems awfully misguided. Even DeepMind doesn't use deep learning for everything.