10 ms·
Is there ever a use case for wanting to compare NON-normalized vectors using dot products or cosine similarity? It seems like all use cases always involve first
by Solvency 3y ago
Is there ever a use case for wanting to compare NON-normalized vectors using dot products or cosine similarity? It seems like all use cases always involve first normalizing the values, which makes sense, but wondering when/why you wouldn't want to do that.
- jstx1 3y agoNormalized dot product is equivalent to cosine similarity.
- esafak 3y agoThe case in which semantic similarity is encoded by Euclidean distance, and you want to perform similarity search (retrieve the nearest neighbors).
- Solvency 3y agoBut in a model like the article examples with 1500+ dimensions doesnt Euclidean distance suffer the "curse of dimensionality", making it unusable for similarity searches?
- esafak 3y agoThat is my understanding. But then again, the angle between of two random points in a high dimensional unit hypercube is 90, so cosine similarity is suspect too. High dimensional spaces are weird like that. https://martin-thoma.com/curse-of-dimensionality/ https://martin-thoma.com/curse-of-dimensionality/
- QuadmasterXLII 3y agoNon-normalized vectors are for when some vectors should match less often in general, encoded by length. In particular, vectors that shouldn’t match anything can be encoded as a zero vector. For example, the last layer of a classification convnet is just dot product similarity, and is usually not normalized