7 ms·
BTW for real world use if you want to do PCA but want a better solution than an algorithm which makes linearity assumptions there are two really hot algorithms
by Der_Einzige 7y ago
BTW for real world use if you want to do PCA but want a better solution than an algorithm which makes linearity assumptions there are two really hot algorithms for dimensionality reduction right now
UMAP - topology manifold learning based method
Ivis - simese triplet network autoencoder
Both of them will blow PCA out of the water on basically all datasets. PCAs only advantages are speed and interpretability (easy to see explained covariance)
- toxik 7y agoPCA is simply Eigen vector extraction on covariance matrix. While more impressive techniques exist, PCA is so simple it will never be passé.
- beagle3 7y agoIndeed. And e.g. generalized eigen problem extend this to the case of two competing data sets. Ignoring the eigen aspect would miss a lot of both theory and practice.
- rectangletangle 7y agoAgreed, there's something to be said for simple models that are "good enough," especially when their limitations are clear. k-NN also comes to mind.
- amrrs 7y agoI'm just wondering if leaving out tSNE was intentional? I'm a big fan of UMAP too. Just created this ^1 Kaggle Kernel a couple of days back to show how UMAP works on Kannada MNIST data. I also think PCA still rules in a lot of cases like when people try to create a new index representative of a bunch of numeric variables [1] https://www.kaggle.com/nulldata/umap-dim-reduction-viz-on-kannada-mnist https://www.kaggle.com/nulldata/umap-dim-reduction-viz-on-ka...
- nestorD 7y agoI am not the OP but I would say that UMAP is a clear superset of t-SNE being able to do the same thing but faster and with a better conservation of large scale distances.
- beagle3 7y agot-SNE doesn’t really tell you how to apply its transformation to new data. (UMAP and PCA do). As a result, t-sne is great for visualization or for “we know everything from the start” data, but not as useful in most data processing pipelines.
- autokad 7y agoNice kernel, hope to use this in the short future
- nestorD 7y agoPCA has speed and interpretability on its side but also the ability to easily add new points (UMAP has a quick fix to do so but its a hack) and go back and forth between the original space and the compressed space making it usable on a wide range of use cases.
- deepnotderp 7y agoYou can also use a nonlinear transform e.g. Gaussianization
- breck 7y ago> PCAs only advantages are speed and interpretability Um, these seem to be perhaps the 2 most important advantages something could have? What are advantages that UMAP and Ivis have that are more important than speed and interpretability?
- nestorD 7y agoNot competent on Ivis but UMAP is very good at making sure that points that are close in the original space are close in the output space. When you want to perform some sort of clustering on top of the dimensionality reduction its a very useful property. Note that is it very common to get rid of 90% of the dimensionality with a PCA and then to apply something like UMAP on top to improve speed (and because PCA can be trusted to keep the relevant information for later postprocessig which cannot be said of other techniques).