There's one important aspect of search engine evaluations that many people forget. A key part in user satisfaction are re-searches (up to 50%, if I recall corr

There's one important aspect of search engine evaluations that many people forget. A key part in user satisfaction are re-searches (up to 50%, if I recall correctly from a paper from Teevan et al.) This means that people expect a search engine to find the same results they found in the past for a given query. That's a huge part of lock-in in the ecosystem.