4 ms·
Linear Methods (PCA) vs. Deep Learning (Autoencoder)
- StefanKarpinski 13y agoNice post, Dan. I'm a sucker for linear methods myself, but I do think they tend to be way less good for classification than for regression and inference (e.g. collaborative filtering). The LSA example is a bit unfair since you're clearly going to need the number of dimensions to be proportional to the number categories for that to work at all well (and if you need to figure out the number of dimensions to use, a scree plot should help). The ability of the deep learning approach to get good separation in only two dimensions is very impressive.
- Irishsteve 13y agoPretty cool to see the viz. I've never found a great improvement with LSI / PCA on text if trying to classify information. A lot of the work in IR seems to suggest improvements in NLP don't improve various ranking models (TF,BM,LM). I wonder do people just revert to lsi / plsi / pca because its fast to compute?