I wish everyone would stop saying that PCA is about eigenvectors and eigenvalues... it's about singular vectors and singular values. The best way to compute PCA

I wish everyone would stop saying that PCA is about eigenvectors and eigenvalues... it's about singular vectors and singular values. The best way to compute PCA is NOT to compute the eigenvalues of A'*A. There are specialised SVD algorithms that should be used instead: https://en.wikipedia.org/wiki/Singular_value_decomposition#Calculating_the_SVD https://en.wikipedia.org/wiki/Singular_value_decomposition#C... Randomised methods have also gotten popular in recent years: https://code.google.com/p/redsvd/ https://code.google.com/p/redsvd/ http://arxiv.org/abs/0909.4061 http://arxiv.org/abs/0909.4061 This is one case where the machine learning community takes a numerical analysis idea and by giving it a different name also loses some of the insights associated with the other name. Normally I don't care if they call it "training" instead of "line search" or "features" instead of "dimensions", but PCA should be using better algorithms. Instead of telling everyone to just use the inferior method of using generic eigenvalue solvers, use better specialised methods. I know that finding eigenvalues of A'A is "good enough", and I guess "eigenvalue" is already a big enough word for most programmers, but there's no reason to not relegate better methods to specialised libraries and make those specialised libraries available. If nothing else, LAPACK and ARPACK have bindings for almost every language.