3 ms·

Yes I had a look at it, I haven't tried e5-mistral-7b-instruct yet but I'll definitely give it a go. Is there such a leaderboard only focused on retrieval by an

• dmezzetti 3y ago
  The BEIR project might be what you're looking for: https://github.com/beir-cellar/beir/wiki/Leaderboard https://github.com/beir-cellar/beir/wiki/Leaderboard
• viraptor 3y ago
  It's interesting how the massive model of e5-mistral has only marginal performance gains over the bge-base and similar ones. It could still be useful for the longer sentence length though.
  • mahjongmen1998 3y ago
    e5-mistral is essentially a distillation from gpt-4 to a smaller model. You can see here https://github.com/microsoft/unilm/blob/16da2f193b9c1dab0a692c6e4380bd43e70a40cd/e5/utils.py#L93 https://github.com/microsoft/unilm/blob/16da2f193b9c1dab0a69... they actually have custom prompts for each dataset being tested. Question would be, if you haven't seen the task before, what is a good prompt to prepend for your task? IMO e5-mistral is overfit to MTEB
• pzar 3y ago
  https://huggingface.co/spaces/mteb/leaderboard https://huggingface.co/spaces/mteb/leaderboard see the retrieval tab