I've been wondering the same. When I dabbled with latent semantic indexing[1], using cosine similarity made sense as the dimensions of the input vectors were w

I've been wondering the same. When I dabbled with latent semantic indexing[1], using cosine similarity made sense as the dimensions of the input vectors were words, for example a 1 if a word was present or 0 if not. So one would expect vectors that point in a similar direction to be related. I haven't studied LLM embedding layers in depth, so yeah been wondering about using certain norms[2] instead to determine if two embeddings are similar. Does it depends on the embedding layer for example? Should be noted it's been many years since I learned linear algebra, so getting somewhat rusty. [1]: https://en.wikipedia.org/wiki/Latent_semantic_analysis https://en.wikipedia.org/wiki/Latent_semantic_analysis [2]: https://en.m.wikipedia.org/wiki/Norm_(mathematics) https://en.m.wikipedia.org/wiki/Norm_(mathematics)