5 ms·

The dot product isn’t a distance

by sfpotter 3y ago

tipsytoad 3y ago
Dot product is equivalent to cosine similarly for normalized vecotrs
- wongarsu 3y ago
  Given normalized vectors, it's also proportional to the square of the Euclidian distance. Meaning for ranking (and thus k-nearest-neighbors) it doesn't matter if you use Euclidian distance or cosine similarity, you get the same result.
- sfpotter 3y ago
  It isn't equivalent to it. See my other comment.
  - tipsytoad 3y ago
    Dude. Cos(x, y) = x . y / ||x|| * ||y|| If x and y are normalized: Cos(x, y) = x . y
    - sfpotter 3y ago
      x*y/|x||y| is a very different function from x*y It also isn't a distance...
deleted 3y ago
[deleted]
mathisfun123 3y ago
My favorite kind of pedantry is pedantry that's naive - on like day 1 of topology 101 you learn that the "dot" product is an inner product and every inner product naturally induces a metric so every inner product space is a metric space (with the induced metric).
- cochne 3y ago
  It’s not pedantry. Just because an inner product induced a metric space does it make it the same thing as that metric. The performance of similarity searches can depend very much on whether you use a dot product or a distance.
- sfpotter 3y ago
  A distance is a function d(x, y) where d(x, y) = 0 if x = y, if x = 0, or if y = 0 (among other properties). If x = y and neither x = 0 nor y = 0, and if we let d(x, y) = x*y, then d(x, y) != 0 in general, so the dot product is certainly not a distance. Your comment isn't relevant, but thanks for calling me a pedant. ;-)
  - mathisfun123 3y ago
    `is != induces`
marginalia_nu 3y ago
It is for unit vectors.
- Solvency 3y ago
  My math is weak. I thought dot product measures alignment. Help me grok what "distance" means here?
  - mecsred 3y ago
    Unit vectors have a normalized length, so the only difference between two unit vectors is the "heading". The angle between them can be interpreted as the the "difference" or distance, between them. Since you can compose any vector in the space out of unit vectors, you can extend the concept. See above comment inner product -> metric space.
  - marginalia_nu 3y ago
    Hmm, I think I got my math wrong too. What is a proper distance function is (1 - |a*b|). That's proportional to the distance between those points on the plane spanned along the circumference of the circle/sphere/hypersphere as it's projected onto that plane.
- sfpotter 3y ago
  The dot product of two unit vectors is the cosine of the angle between them. Since cos^-1 (u*v) is the angle between them, or the geodesic distance between u and v regarded as points on the sphere, which is a distance. But the dot product is not. If u = v, then u*v = 1, but for two points not equal to zero, the distance between them should always be zero. Remember that u*v = 0 if u and v are perpendicular. If neither u nor v are 0, then for them to be perpendicular, they must certainly not be equal. The dot product does not behave like a distance.

The dot product isn’t a distance

tipsytoad 3y ago

Dot product is equivalent to cosine similarly for normalized vecotrs

wongarsu 3y ago
Given normalized vectors, it's also proportional to the square of the Euclidian distance. Meaning for ranking (and thus k-nearest-neighbors) it doesn't matter if you use Euclidian distance or cosine similarity, you get the same result.
sfpotter 3y ago
It isn't equivalent to it. See my other comment.
- tipsytoad 3y ago
  Dude. Cos(x, y) = x . y / ||x|| * ||y|| If x and y are normalized: Cos(x, y) = x . y
  - sfpotter 3y ago
    x*y/|x||y| is a very different function from x*y It also isn't a distance...

deleted 3y ago

[deleted]

mathisfun123 3y ago

My favorite kind of pedantry is pedantry that's naive - on like day 1 of topology 101 you learn that the "dot" product is an inner product and every inner product naturally induces a metric so every inner product space is a metric space (with the induced metric).

cochne 3y ago
It’s not pedantry. Just because an inner product induced a metric space does it make it the same thing as that metric. The performance of similarity searches can depend very much on whether you use a dot product or a distance.
sfpotter 3y ago
A distance is a function d(x, y) where d(x, y) = 0 if x = y, if x = 0, or if y = 0 (among other properties). If x = y and neither x = 0 nor y = 0, and if we let d(x, y) = x*y, then d(x, y) != 0 in general, so the dot product is certainly not a distance. Your comment isn't relevant, but thanks for calling me a pedant. ;-)
- mathisfun123 3y ago
  `is != induces`

marginalia_nu 3y ago

It is for unit vectors.

Solvency 3y ago
My math is weak. I thought dot product measures alignment. Help me grok what "distance" means here?
- mecsred 3y ago
  Unit vectors have a normalized length, so the only difference between two unit vectors is the "heading". The angle between them can be interpreted as the the "difference" or distance, between them. Since you can compose any vector in the space out of unit vectors, you can extend the concept. See above comment inner product -> metric space.
- marginalia_nu 3y ago
  Hmm, I think I got my math wrong too. What is a proper distance function is (1 - |a*b|). That's proportional to the distance between those points on the plane spanned along the circumference of the circle/sphere/hypersphere as it's projected onto that plane.
sfpotter 3y ago
The dot product of two unit vectors is the cosine of the angle between them. Since cos^-1 (u*v) is the angle between them, or the geodesic distance between u and v regarded as points on the sphere, which is a distance. But the dot product is not. If u = v, then u*v = 1, but for two points not equal to zero, the distance between them should always be zero. Remember that u*v = 0 if u and v are perpendicular. If neither u nor v are 0, then for them to be perpendicular, they must certainly not be equal. The dot product does not behave like a distance.