Ok I had some sleep. I agree with all of your points except the measurement issue. Yes you can compare Search Engines the way you described, but just because of

Ok I had some sleep. I agree with all of your points except the measurement issue. Yes you can compare Search Engines the way you described, but just because of the size of the underlying corpus, the web, there is no way to measure the goodness of a system over the whole corpus. And yeah my post was a bit brash, subjective opinion typed in by a guy who really lacked some sleep. :-)