4 ms·
How do you people who work with high-dimensional data outside of biology feel about t-SNE and/or UMAP? Some of the points against t-SNE feel like comments that
by ejstronge 7y ago
How do you people who work with high-dimensional data outside of biology feel about t-SNE and/or UMAP?
Some of the points against t-SNE feel like comments that only non-computer scientists would make (e.g., t-SNE must be run on a cluster/needs a lot of RAM - despite the fact that rather few genetic datasets can be analyzed on the commodity laptops most common among biologists).
- Der_Einzige 7y agoUMAP is basically one of the most innovative things I've ever seen. It's highly used in NLP with extremely good results
- throwaway66920 7y agoExample? Anything besides visualizing embedding spaces?
- Der_Einzige 7y agoAny type of clustering
- lmeyerov 7y agoWe've been doing it increasingly w/ our users at Graphistry. Think security events, fraud accounts, etc. The reason we do graph to beginwith is to help explore relationships & correlations in data in an explainable & interactive way, and GPU-accelerated UMAP over interactive GPU visual analytics helps us bring a bit of that to metadata that doesn't work as well as a graph (continuous dimensions vs. categorical). Loading in 1M flagged accounts or events and visually drilling in is pretty magical :) Explainability is still worse than user experiences with our graph and temporal methods, and both t-sne/umap struggle on categorical (the most important aspects of most operational data). The good news is neither problem seems inherent. UMAP is definitely on the shortlist of methods we've been tracking here. I'm hoping to demo some externally accessible IT examples & workflows here by end-of-month. This stuff is very cool, so we're aiming to make tools here easier / more reliable / more practical for operational teams.
- nerdponx 7y agoUMAP has a reputation for being less "fussy" with parameter choices.