Nice post, Dan. I'm a sucker for linear methods myself, but I do think they tend to be way less good for classification than for regression and inference (e.g.

Nice post, Dan. I'm a sucker for linear methods myself, but I do think they tend to be way less good for classification than for regression and inference (e.g. collaborative filtering). The LSA example is a bit unfair since you're clearly going to need the number of dimensions to be proportional to the number categories for that to work at all well (and if you need to figure out the number of dimensions to use, a scree plot should help). The ability of the deep learning approach to get good separation in only two dimensions is very impressive.