The "Web track" task at the annual US NIST TREC conference ( https://trec.nist.gov/ https://trec.nist.gov/ ) is an open innovation benchmark that everyone can

The "Web track" task at the annual US NIST TREC conference ( https://trec.nist.gov/ https://trec.nist.gov/ ) is an open innovation benchmark that everyone can contribute; participants get a set of queries that they have to run on exactly the same corpus. Then they return the top-k results to a team that evaluates them. Here is an example (2014) Web track paper of the 23th TREC: https://trec.nist.gov/pubs/trec23/papers/overview-web.pdf https://trec.nist.gov/pubs/trec23/papers/overview-web.pdf (TREC has a plentitude of difference benchmark tasks and you can submit your own: https://trec.nist.gov/pubs/trec29/trec2020.html https://trec.nist.gov/pubs/trec29/trec2020.html - recent TREC 2020 papers)