3 ms·
It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned abov
by genmon 4y ago
It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned above.
It's weird -- it's locally useful to navigate, and at a high level kinda useful, but only if you squint and don't look at the problems. So I feel like a fisheye visualisation would be appropriate? That's something that I'm exploring in other projects.
- gwern 4y agoI wouldn't necessarily reach for PCA. No reason to think that the first two principal components necessarily encode anything particularly interesting. If you want to lay out each point in 2D in a way which keeps similar points nearby, something like t-SNE is worth a try - visualizing embeddings is what it was invented for.
- genmon 4y agoExcellent, new to me and I'll give it a go, thanks! I gravitate to PCA for terrible reasons (undergrad so it's what I think of first) and like you say, it's beguiling yet disappointing, the components rarely have any human meaning.
- pigscantfly 4y agoI'd suggest trying t-SNE [1] instead; you'll be losing almost all of the variance by projecting onto the first two eigenvectors produced by PCA. [1] http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-with-t-sne-in-Javascript/ http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-...
- nl 4y agoTSNE or UMAP (as others have mentioned) is good but also take a look at the tensorflow projector. You can host your own and/or pass custom data to it.
- PeterisP 4y agoThe word game of 'semantle' (https://semantle.com/ https://semantle.com/) is a nice way to get an intuitive grasp of how unintuitive the closeness in a highdimensional space is, as you're required to guess a word based on it's semantic similarity (according to classic word2vec metric) to your previous guesses.