3 ms·
That would be great. However if you wanted to benchmark relevance ranking, how would you do that?
by jasfi 4y ago
That would be great. However if you wanted to benchmark relevance ranking, how would you do that?
- sanxiyn 4y agoYou need a dataset and an evaluation metric. The usual evaluation metric is NDCG(Normalized Discounted Cumulative Gain): https://en.wikipedia.org/wiki/NDCG https://en.wikipedia.org/wiki/NDCG An example dataset is BEIR(BEnchmarking Information Retrieval), published in NIPS 2021: https://github.com/beir-cellar/beir https://github.com/beir-cellar/beir