Mr Shipilev is pretty well aware of this. He used to give a talk "JMH: The Lesser of Two Evils" that goes into some detail about a lot of the problems and pitfa

Mr Shipilev is pretty well aware of this. He used to give a talk "JMH: The Lesser of Two Evils" that goes into some detail about a lot of the problems and pitfalls with microbenchmarks. The argument isn't that microbenchmarks are never useful, but rather that they are usually not very applicable and extremely hard to get right, and even harder to interpret. If you're doing JVM optimizations, it may be a useful tool, but like in practice, it's questionable whether these 12x optimizations will result in even a 0.1% improvement in performance in the median Java application. It's pretty simple when you think about it. If you're optimizing effects that are hard to isolate and in many ways barely measurable in the noise of process memory alignment and similar factors, then in most cases, whatever optimization you make is likewise going to be barely measurable in the noisy hardware/software landscape.