10 ms·

Thanks! 1536 That's quite a lot. Is there a fast way to find the most similar one to a given one in SQLite? cosine similarity Why not euclidean dis

• moconnor 4y ago
  Very high-dimensional space is unintuitive. It can help to imagine a cube as N increases - 2^N corners and vanishingly small volume to surface area. This gives useful intuitions e.g. cosine distance ignores length and only looks at angles, but this matters less with higher dims; by contrast the euclidian length becomes less useful.
  • TekMol 4y ago
    Do I understand you correctly, that you say cosine distance is a better measurement of similarity than euclidian distance when there are many dimensions?
    • deleted 4y ago
      [deleted]
  • sebzim4500 4y ago
    I get that cosine distance is easier to calculate, but isn't sorting by cosine distance the same thing as sorting by L2 norm, assuming that the vectors are already L2 normalized?
    • deleted 4y ago
      [deleted]
• Shorel 4y ago
  They use cosine because of performance reasons. Euclidean distance when used is squared, because even that sqrt is too expensive.
  • iandanforth 4y ago
    Also note that the embeddings are learned based on which distance metric is used, thus making it important to use the right distance metric with the right embeddings above and beyond wall-clock performance. Other metrics may still work, but less well.
• deleted 4y ago
  [deleted]