sigh. The amount of work it takes to produce a really good benchmark is something more than what this guy did. Exhibit A: his analysis of optimization impact.

sigh. The amount of work it takes to produce a really good benchmark is something more than what this guy did. Exhibit A: his analysis of optimization impact. I've been accused of biasing the results by using the -O2 option for GCC, supposedly because -O2 optimizes for space, thus slowing down the benchmark. This is not what -O2 does. According to the GCC -O documentation... -O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code. On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think -O2 was the best choice. Uh...what about optimizations performed in the JVM (or, more specifically, the JIT)? How do they compare? How do you know that -O2 is the right choice, other than "it's kinda in the middle between lots of optimizations and not many, so that sounds good."? He could have started with this: http://java.sun.com/j2se/1.4.2/1.4.2_whitepaper.html http://java.sun.com/j2se/1.4.2/1.4.2_whitepaper.html Exhibit B: he may have introduced a ton of bias through test selection and doesn't even know it Some of the C++ tests would not compile. I've never been very good at decoding GCC's error messages, so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks. What?? If you can't read GCC's error messages, I don't get any warm fuzzies about your ability to decide which tests compare the things you want to compare or not. There's plenty more to be critical of here, but the above two really were the sore thumbs in this article. /me gets back to real work and plans to dissect this more tonight