4 ms·
PCA goes far deeper than meets the eye. For instance, it's a well-known phenomenon that too much dimensionality can actually drive predictor performance to rand
by tel 14y ago
PCA goes far deeper than meets the eye. For instance, it's a well-known phenomenon that too much dimensionality can actually drive predictor performance to random, but PCA can mitigate that. It's a basically the bread and butter of practical unsupervised learning.
- mturmon 14y ago"bread and butter of practical unsupervised learning" -- true, although I might have said "exploratory data analysis". If you can make a vector out of it somehow, it can't hurt to try PCA. Because you don't have to figure out some fancy tailored model, or really (cough, cough) understand much about the data at all. (It sounds like I'm being sarcastic, but I'm serious -- sometimes all you want is a quick look.) Unsupervised clustering is a similar technique.
- tel 14y agoI find that more often than expected, PCA (or maybe MDS) gets a majority of the performance of any kind of unsupervised method. If you're really interested in exploring the data and methodologies, then PCA is a poor stopping point... but if you just want it to work, it's surprising how well PCA tradeoffs are good tradeoffs. All the obvious caveats apply to that whole line of thought, though.
- snikolov 14y agoIndeed, projection onto the principal subspace is a kind of regularization, which makes for better generalization. This kind of makes sense intuitively. These slides go into more detail (the whole course is great), where it says "Projection regularizes!" http://www.mit.edu/~9.520/spring09/Classes/class07_spectral.pdf http://www.mit.edu/~9.520/spring09/Classes/class07_spectral....