3 ms·

Question to Y-hat folks: why cluster in 2D? Granted, clustering in 300D is hard :) Still, the 2D projection must add a significant metric distortion. Why not a

by ilyaeck 10y ago

Question to Y-hat folks: why cluster in 2D? Granted, clustering in 300D is hard :) Still, the 2D projection must add a significant metric distortion. Why not a middle ground, say, 5-10D ?

minimaxir 10y ago
Unless you are Mister Mxyzptlk, you cannot plot visualizations greater than 2-3D.
- sixhobbits 10y ago
  parent is asking about this I think: "This is indeed nice for data visualization, while it’s also very helpful in our pipeline because it removes noise in the derived vectors, by forcing a new mapping based purely on relative similarity. For this reason we will be using the low-dimensional coordinates of each word in our recommender system." i.e. it's nice for visualisation, and it removes noise. It would be interesting to see discussion from y-hat about where the sweet spot between lack of noise and still keeping relevant information is. I think because the subject matter is pretty simply to cluster, 2D works well enough and keeps everything simple.
BenoitP 10y ago
Do you mean replacing word2vec -> t-SNE(2D) -> HDBSCAN by word2vec -> t-SNE(10D) -> HDBSCAN -> t-SNE(2D) (or just 2D projection that separates well the centroids) ? ---- About clustering in 300D, specifically for word2vec vectors, to which cosine similarity is important: Random-Projections LSH is supposed to "be compatible" with the cosine similarty. I have been toying with it and getting sort of good recalls (70%) with 100D. It might help a lot with regards to finding the nearest neighbours. Or were you meaning that the clustering itself will be meaningless, regardless of our ability to find neighbours?
lmcinnes 10y ago
Personally I think that would make some sense. I suspect the catch is in presenting the results to the user: you have to present in 2D, and if you cluster in higher dimensions you may get results that, while perfectly valid, look strange when collapsed to 2D (overlapping/mixed clusters etc.) To my mind this isn't so bad -- the notion of overlapping clusters is something I think most people actually accept (think of clusters as tags rather than a partitioning and you can get the idea). However I can see why you may prefer to have your clusters present more clearly in 2D.
dandermotj 10y ago
Multidimensional scaling is designed to preserve the most variance (or some variation of that idea) among points in a lower dimensional space. So for the most part, the cluster memberships assigned by clustering in an n-dimensional space are the same in the lower dimensional projection. There's often no need to cluster in a higher space because the outcome is the same, and clustering is more expensive when there's more dimensions.

by ilyaeck 10y ago

Question to Y-hat folks: why cluster in 2D? Granted, clustering in 300D is hard :) Still, the 2D projection must add a significant metric distortion. Why not a middle ground, say, 5-10D ?

minimaxir 10y ago

Unless you are Mister Mxyzptlk, you cannot plot visualizations greater than 2-3D.

sixhobbits 10y ago
parent is asking about this I think: "This is indeed nice for data visualization, while it’s also very helpful in our pipeline because it removes noise in the derived vectors, by forcing a new mapping based purely on relative similarity. For this reason we will be using the low-dimensional coordinates of each word in our recommender system." i.e. it's nice for visualisation, and it removes noise. It would be interesting to see discussion from y-hat about where the sweet spot between lack of noise and still keeping relevant information is. I think because the subject matter is pretty simply to cluster, 2D works well enough and keeps everything simple.

BenoitP 10y ago

Do you mean replacing word2vec -> t-SNE(2D) -> HDBSCAN by word2vec -> t-SNE(10D) -> HDBSCAN -> t-SNE(2D) (or just 2D projection that separates well the centroids) ? ---- About clustering in 300D, specifically for word2vec vectors, to which cosine similarity is important: Random-Projections LSH is supposed to "be compatible" with the cosine similarty. I have been toying with it and getting sort of good recalls (70%) with 100D. It might help a lot with regards to finding the nearest neighbours. Or were you meaning that the clustering itself will be meaningless, regardless of our ability to find neighbours?

lmcinnes 10y ago

Personally I think that would make some sense. I suspect the catch is in presenting the results to the user: you have to present in 2D, and if you cluster in higher dimensions you may get results that, while perfectly valid, look strange when collapsed to 2D (overlapping/mixed clusters etc.) To my mind this isn't so bad -- the notion of overlapping clusters is something I think most people actually accept (think of clusters as tags rather than a partitioning and you can get the idea). However I can see why you may prefer to have your clusters present more clearly in 2D.

dandermotj 10y ago

Multidimensional scaling is designed to preserve the most variance (or some variation of that idea) among points in a lower dimensional space. So for the most part, the cluster memberships assigned by clustering in an n-dimensional space are the same in the lower dimensional projection. There's often no need to cluster in a higher space because the outcome is the same, and clustering is more expensive when there's more dimensions.