The best way I can think of would be to have a long term test where a number of users did all of their searching through a site that would randomly show the res

The best way I can think of would be to have a long term test where a number of users did all of their searching through a site that would randomly show the results of one search engine, and then have the users rate their satisfaction with the results. It wouldn't be quite as good at cross comparing, but it would have a much more realistic selection of searches to draw from, and it would reduce the sort of biases that the gp post was talking about.