4 ms·
Data was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-s
by Houshalter 9y ago
Data was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-smartphones.
- xapata 9y agoWhen I was in school, studying ML, we were always able to beat NN performance with SVMs on the datasets available. It's only through larger training sets that NNs start to shine.
- Houshalter 9y agoNNs will still be beat by svms on simple datasets, even big ones. Where NNs shine is when the data has an underlying structure that can be exploited. Like how convolutions take advantage of the 2 dimensional nature of images, or RNNs take advantage of sequential data. Algorithmic improvements like dropout and proper weight initialization have also made them bunch better than they used to be.
- xapata 9y agoIf you know what the structure is, then you could write a proper kernel and then SVMs are better again. NNs win when you don't know what the structure is and it's too complex to approximate with a standard kernel.
- Houshalter 9y agoI'm not aware of anything like a convolutional SVM, but I'm not terribly familiar with them. As I understand it SVMs are fundamentally shallow learners, and if you try to hack them to make them deep and recurrent or convolutional, you just get weird NNs.
- xapata 9y agoShallow in the sense that SVMs are not layered, but the layering of a NN is only to enable the humble perceptron (logistic or sigmoid, usually) to model more complex functions. In contrast, the SVM doesn't need to layer, because the kernel can be arbitrarily complex.