It's a good thing that I didn't say anything about "automated algorithms used to generate metrics", then... This was machine-aided human evaluation. Search qu

It's a good thing that I didn't say anything about "automated algorithms used to generate metrics", then... This was machine-aided human evaluation. Search quality evaluation can't really be done without humans in the loop. If you had an algorithm that could distinguish between a good result and a bad result, you wouldn't use it to evaluate results. You'd use it to generate the results.