5 ms·
Another angle: the PCA is given by computing the SVD (a more general analog of eigenvalue/eigenvector decomposition) of a whitened representation of the data.
by thearn4 13y ago
Another angle: the PCA is given by computing the SVD (a more general analog of eigenvalue/eigenvector decomposition) of a whitened representation of the data. Some idiosyncrasies of PCA them become obvious: we can't determine if a computed result is actually a sought result or its reflection/negative, because the SVD is only unique up to sign variance.
This is also closer to it's actual implementation: while it's true that you do technically need the eigenbasis of the covariance matrix, you should not actually form the covariance matrix to get there...
- Radim 13y agoyou should not actually form the covariance matrix to get there... In case anyone's wondering why: it's not only because it takes extra time. The main reason is that computing X*X^T can bring numerical instability, where a direct SVD(X) would work just fine.
- neumann 13y agoI remember this being elegantly demonstrated here: http://math.stackexchange.com/questions/3869/what-is-the-intuitive-relationship-between-svd-and-pca http://math.stackexchange.com/questions/3869/what-is-the-int...
- conjectures 13y agoWith mean corrected observations as rows of a data matrix X, the sample covariance matrix is given by X'X not XX'. Also (X'X)g = lg (XX')Xg = lXg So with n>>p and n large, you may be able to fit X'X (p^2 entries) but not XX' (n^2 entries) in memory (and get the same eigenvalues and related eigenvectors).
- twstws 13y ago> we can't determine if a computed result is actually a sought result or its reflection/negative That's not how I'd explain it. The 'sought result' and its reflection are not two different things. They are the same thing, differing only in a trivial detail of orientation. The 'negative' of the result conveys exactly the same information as the result.