The problem is the increased risk of nonsense benchmarks. I.e. if the unoptimized code is super slow because the naive implementation compiles to something unsu

The problem is the increased risk of nonsense benchmarks. I.e. if the unoptimized code is super slow because the naive implementation compiles to something unsuited to, say, the x86 microarchitecture. Or that performance is limited by lock contention. That's way more likely with pre-release then production-grate software. If you want to compare B1 and B2 that's great - but then you post to hackernews a comparison between two Drupal betas, not a comparison between two hardware platforms.