If the benchmarks were the kind of bad that you allude to in your second statement it would probably be pretty reasonable for the reasons you state, and overall

If the benchmarks were the kind of bad that you allude to in your second statement it would probably be pretty reasonable for the reasons you state, and overall you would probably end up with a distribution of benchmarks that approximated overall performance. What about intentionally crafted, nigh pathological examples? With the kind of flaws that don't crop up in real world code? I don't think those have value. Actually, I think I'm trying to distinguish between flawed benchmarks like you are talking about and actively malignant benchmarks.