5 ms·
Pardon my ignorance, but are you saying that embedding model leaderboards aren't useful? When I look at https://huggingface.co/spaces/mteb/leaderboard https://h
by moojacob 2y ago
Pardon my ignorance, but are you saying that embedding model leaderboards aren't useful? When I look at https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard I see voyage's embedding model is at the top. Does that mean they have the best model?
- fzliu 2y agoAlthough voyage-3-m-exp is at the top of the leaderboard, I would not use it for production use cases unless your data is extremely similar to "in-domain" MTEB data. Check out this post from Nils (one of the authors of the original MTEB paper) for more info: https://x.com/Nils_Reimers/status/1870812625505849849 https://x.com/Nils_Reimers/status/1870812625505849849 voyage-3-large will work better for almost all real-world production use cases.
- moojacob 2y agoWhich model would you choose?
- evilduck 2y agoThey did not claim that leaderboards were useless. There’s a big difference between an embedding model leaderboard and the SWEbench leaderboard though, given that they measure completely different categories of models.