4 ms·
This is good, but could also be good to mention that you're using umap for dimensionality reduction with cosine metric. https://github.com/Z-Gort/Reservoirs-La
by wenc 2y ago
This is good, but could also be good to mention that you're using umap for dimensionality reduction with cosine metric.
https://github.com/Z-Gort/Reservoirs-Lab/blob/main/src/electron/scripts/reduceDimension.py https://github.com/Z-Gort/Reservoirs-Lab/blob/main/src/elect...
Dimensionality reduction from n >> 2 dimensions to 2 dimensions can be very fickle, so the hyperparameters matter. Your visualization can change significantly significantly depending on choice of metric.
https://umap-learn.readthedocs.io/en/latest/parameters.html https://umap-learn.readthedocs.io/en/latest/parameters.html
You may want to consider projecting to more than 2 dimensions too. You may ask, how does one visualize more than two dimensions? Through a scatterplot matrix of 2 axes at a time.
https://seaborn.pydata.org/examples/scatterplot_matrix.html https://seaborn.pydata.org/examples/scatterplot_matrix.html
These are used for PCA-type multivariate analyses to visualize latent variables in higher dimensions than 2, but 2 dimensions at a time. Some clustering behavior that cannot be seen in 2 axes might be seen in higher dimensions. We used to do this our lab to find anomalies in high dimensions.
- isoprophlex 2y agoAbout fickleness... indeed i've found this a kinda problematic thing when running large-d text embeddings through umap -- it always comes out spherical, blob-shaped, without any obvious segregation in the low-d projected space. IMO it's very difficult to make a "fire and forget" embedding interpreter. Maybe I never found the right parameters to umap but the results of running it (or any dimension reduction algo) always left me a bit underwhelmed.
- antman 2y agoHave you tried PaCMAP? It should be better and faster
- wenc 2y agoThanks for the pointer to PacMap. I just tried it. My verdict? PacMap >= UMAP >> t-SNE. UMAP captures the basic pattern but PacMap makes it crisper.
- isoprophlex 2y agoWow, thanks for that!