That's true but it also allows me to look for selection bias in their data. If I see a paper claiming remarkably good predictive capabilities of (say) the perf

That's true but it also allows me to look for selection bias in their data. If I see a paper claiming remarkably good predictive capabilities of (say) the performance of a basic block, and no discussion of it's flaws you bet I'm assuming they didn't test it well enough.