2 ms·
Deep learning is machine learning involving neural nets with more than one hidden layer.
by taneq 10y ago
Deep learning is machine learning involving neural nets with more than one hidden layer.
- tomp 10y ago... and a method to train the hidden layers efficiently.
- taneq 10y agoWell, therein lies the rub, yes.
- _delirium 10y agoThat's arguable; most successful applications of deep learning aren't particularly efficient on the training side, which is why they come out of research groups that have Google-sized compute clusters available, typically with big arrays of recent GPUs. I suppose they're efficient in the sense of not requiring more computation than exists on earth. There have been a bunch of algorithmic advances as well, but some advances have simply been those increases in compute power: there are techniques that didn't really work in 1985, where the technique or a close variant now works, mainly because we didn't the big arrays of GPUs in 1985 that we do now. As these get scaled up, the algorithmic advances are being found to not even be necessary in some cases. For a period it was thought that autoencoder pretraining was the big algorithmic breakthrough that made it feasible to train many-layered neural networks, but a number of recent applications no longer bother with the autoencoder pretraining.
- Kip9000 10y agoTo do increasingly fancy things with feed forward neural networks, we require increasingly many layers. But with many layers, it becomes increasingly difficult to train it (vanishing gradient, explosive growth of connections, etc). But Hinton et al. found a few tricks to go past those obstacles and that's what 'deep learning' encompasses. Current state of the art of deep learning is at 150 odd layers with a new trick called 'residual learning' by Microsoft Research China.
- taneq 10y agoI didn't say it was easy to do, if anything I was just taking a pot shot at the laziness of using the term as a label for all of modern machine learning. I've been studying up on deep neural nets recently though and there are a bunch of very cool tricks (and I don't mean that pejoratively, I think real intelligence consists of just such a grab-bag of tricks), which make it effective in a wide range of cases.
- nabla9 10y agoMost definitions of deep learning don't include world "neural network". They are synonymous for multi layer neural network techniques for now, but not necessarily in the future. For example: >Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. Deep Learning, Nature Volume 521 issue 7553, 2015, LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey