LLaMA embeddings would perform terribly. MS MARCO is a well known benchmark that mirrors a pretty common use case: https://www.sbert.net/docs/pretrained-models

LLaMA embeddings would perform terribly. MS MARCO is a well known benchmark that mirrors a pretty common use case: https://www.sbert.net/docs/pretrained-models/msmarco-v3.html https://www.sbert.net/docs/pretrained-models/msmarco-v3.html https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard To intentionally oversimplify, embeddings fall into 2 main categories if you do a cursory search: - Good at semantic similarity: Most common case, check if two strings have similar meaning, even if the words don't match exactly. - Good at Q/A: Finds text that can answer a question. Sound similar to semantic, but "What is a dog" and "What is a cat" are very similar sentences but very different questions. These models cluster questions closer to their answers and further from other questions. They also handle the difference in length between questions and answers better. (Some leading embedding models use prefixes during training to let you adjust performance between those two tasks on the fly) LLaMA embeddings weren't optimized to any specific task, you'd just be hoping that they tangentially align with some arbitrary goal. OpenAI had to fine tune their embeddings model, and despite being massively oversized you can see it's not at the top of the leaderboard compared to much smaller models.