3 ms·
> The "embedding" for a text is an array of floating point numbers? Yes, that's what an embedding is. > How many? It varies depending on the model. In this c
by jstx1 4y ago
> The "embedding" for a text is an array of floating point numbers?
Yes, that's what an embedding is.
> How many?
It varies depending on the model. In this case 1536, it's in the original post.
> And how do you calculate the distance between two texts then?
You take their embedding vectors and calculate the distance between the two vectors using something like cosine similarity - https://en.wikipedia.org/wiki/Cosine_similarity https://en.wikipedia.org/wiki/Cosine_similarity (there are other distances too obviously)
- TekMol 4y agoThanks! 1536 That's quite a lot. Is there a fast way to find the most similar one to a given one in SQLite? cosine similarity Why not euclidean distance?
- moconnor 4y agoVery high-dimensional space is unintuitive. It can help to imagine a cube as N increases - 2^N corners and vanishingly small volume to surface area. This gives useful intuitions e.g. cosine distance ignores length and only looks at angles, but this matters less with higher dims; by contrast the euclidian length becomes less useful.
- TekMol 4y agoDo I understand you correctly, that you say cosine distance is a better measurement of similarity than euclidian distance when there are many dimensions?
- deleted 4y ago[deleted]
- sebzim4500 4y agoI get that cosine distance is easier to calculate, but isn't sorting by cosine distance the same thing as sorting by L2 norm, assuming that the vectors are already L2 normalized?
- deleted 4y ago[deleted]
- Shorel 4y agoThey use cosine because of performance reasons. Euclidean distance when used is squared, because even that sqrt is too expensive.
- iandanforth 4y agoAlso note that the embeddings are learned based on which distance metric is used, thus making it important to use the right distance metric with the right embeddings above and beyond wall-clock performance. Other metrics may still work, but less well.
- deleted 4y ago[deleted]