3 ms·

I'm surprised the warmup problem in Java hasn't been invoked here yet. Java has bootstrap and warm-up overhead which for comparisons like this one may have a de

I'm surprised the warmup problem in Java hasn't been invoked here yet. Java has bootstrap and warm-up overhead which for comparisons like this one may have a decisive impact on the perceived results. What will happen is (after the JVM starts), there will be (by default) 10000 interpreted iterations, then shortly after that a stop-the-world pause to replace slow interpreted code with its native counterpart JITed by a compiler thread, and probably a few more of those either optimising other, less hot parts of the code, or applying more aggressive optimizations on already JITed sections. Go comes with AOT compilation and as far as I know, doesn't have much initial overhead. For a test that takes few seconds to complete, I'd say this is not how you want to compare performance.

hyperpape 10y ago
What you've said about the JVM overhead is true, but what the experiment showed is that on the time span of six seconds, implementing an algorithm that's this simple, the overhead does not result in the Java solution being slower than the Go solution. Full native code compilation for a hot method can be triggered and completed in a fraction of a second after startup. The net result is that for algorithmic problems that finish in seconds rather than milliseconds, the JVM startup cost may not be an issue. This will only be true if, like this problem, you're loading a small number of classes. Most people's experience with crappy JVM startup time probably has a lot to do with with web application containers like Tomcat that load thousands of classes in the first five seconds. Probably a good test to do to assess real-world performance would be something that does file-system operations and employs some parsing libraries, rather than just building data structures in memory and using System.out. You're correct that if you want to compare peak performance, you should run the tests differently, so that you're not measuring the JVM startup cost.
jnordwick 10y ago
There is no stop the world point. There are compilation threads always running in the background patching hot code. In Java 8 tiered compilation became the default. After a first threshold is reached, a method or loop will be compiled as a first approximation with further record keeping. At the second threshold, a more aggressively optimized version without record keeping will be put in place. This is kind of like the merging of the old C1 and C2 compilers into one to help get up to speed quicker. When you want to measure this stuff you use JMH, Java Microbenchmarking Harness, an amazing piece of work that will warmup the JVM before executing performance tests. Be very careful writing your own Java timing code, and for the most part rely on JMH.

hyperpape 10y ago

What you've said about the JVM overhead is true, but what the experiment showed is that on the time span of six seconds, implementing an algorithm that's this simple, the overhead does not result in the Java solution being slower than the Go solution. Full native code compilation for a hot method can be triggered and completed in a fraction of a second after startup. The net result is that for algorithmic problems that finish in seconds rather than milliseconds, the JVM startup cost may not be an issue. This will only be true if, like this problem, you're loading a small number of classes. Most people's experience with crappy JVM startup time probably has a lot to do with with web application containers like Tomcat that load thousands of classes in the first five seconds. Probably a good test to do to assess real-world performance would be something that does file-system operations and employs some parsing libraries, rather than just building data structures in memory and using System.out. You're correct that if you want to compare peak performance, you should run the tests differently, so that you're not measuring the JVM startup cost.

jnordwick 10y ago

There is no stop the world point. There are compilation threads always running in the background patching hot code. In Java 8 tiered compilation became the default. After a first threshold is reached, a method or loop will be compiled as a first approximation with further record keeping. At the second threshold, a more aggressively optimized version without record keeping will be put in place. This is kind of like the merging of the old C1 and C2 compilers into one to help get up to speed quicker. When you want to measure this stuff you use JMH, Java Microbenchmarking Harness, an amazing piece of work that will warmup the JVM before executing performance tests. Be very careful writing your own Java timing code, and for the most part rely on JMH.