4 ms·
What is the place of support vector machines once deep learning techniques are going mainsteeam?
by wslh 14y ago
What is the place of support vector machines once deep learning techniques are going mainsteeam?
- textminer 14y agoAm I crazy, or is the short-run benefit more for unsupervised feature learning more than classification accuracy? If I recall from my brief reading on the subject, kernel-aided SVMs and Random Forests are basically equivalent to a three-layer deep graphical net, but on painfully feature-engineered inputs. I'd wager learning features is more useful now than any benefit of a super-deep architecture. (I would love for a practitioner to shoot this down or corroborate my hunch-- awfully new to neural nets, having basically read a monograph, played with Theano, and watched some Hinton lectures.)
- ihodes 14y agoWhat did you read, and would you recommend it? I'm looking for some good reading on the subject.
- awc 14y agoSVMs may still be useful as a classifier sitting on top of a deep model. Plugging the features learned during unsupervised pre-training into an SVM instead of a neural net is perfectly valid. Lower layers of the "deep" model are (typically) performing non-linear dimensionality reduction. i.e. generating a set of high-level features which make subsequent classification easier than it would have been on the raw input data.
- fiatmoney 14y agoYou can use SVMs for deep learning, for one. http://books.nips.cc/papers/files/nips25/NIPS2012_1290.pdf http://books.nips.cc/papers/files/nips25/NIPS2012_1290.pdf