3 ms·
PCA is nice if you know relationships are linear. You also want to be aware of TSNE and UMAP.
by baq 1y ago
PCA is nice if you know relationships are linear. You also want to be aware of TSNE and UMAP.
- wenc 1y agoA lot of relationships are (locally) linear so this isn’t as restrictive as it might seem. Many real-life productionized applications are based on it. Like linear regression, it has its place. T-SNE is good for visualization and for seeing class separation, but in my experience, I haven’t found it to work for me for dimensionality reduction per se (maybe I’m missing something). For me, it’s more of a visualization tool. On that note, there’s a new algorithm that improves on T-SNE called PaCMAP which preserves local and global structures better. https://github.com/YingfanWang/PaCMAP https://github.com/YingfanWang/PaCMAP
- a_bonobo 1y agoThere's also Bonsai, it's parameter-free and supposedly 'better' than t-SNE, but it's clearly aimed at visualisation purposes (except that in Bonsai trees, distances between nodes are 'real' which is usually not the case in t-SNE) https://www.biorxiv.org/content/10.1101/2025.05.08.652944v1.full https://www.biorxiv.org/content/10.1101/2025.05.08.652944v1....
- energy123 1y agoI'd add that PCA/OLS is linear in the functional form (linear combination), but the input variables can be non-linear (X_new := X_{old,1}*X_{old,2}^2), so if the non-linearities are simple, then basic feature engineering to strip out the non-linearities before fitting PCA/OLS may be acceptable.