4 ms·

Have you tried setting up a JMH benchmark? This should allow you to see if the JIT is the cause of your slowdowns. Also, running it under a profiler (I recommen

Have you tried setting up a JMH benchmark? This should allow you to see if the JIT is the cause of your slowdowns. Also, running it under a profiler (I recommend async-profiler[1]) should give you a good idea of where the slowdown occurs which might help you pin it down further. [1] https://github.com/jvm-profiling-tools/async-profiler https://github.com/jvm-profiling-tools/async-profiler

winrid 5y ago
Would love to see the output of this if someone does it. Sometimes things like this are caused by your code being "un-jitted", in the event some criteria is met, like an exception being thrown for example (although BitSet does not look to do that internally).
pvg 5y ago
This is probably the first thing the author should try. Just a quick look at their own harness, it's sort of trying to guess what 'warmup' is required. The JIT kicks in at invocation counts, among other things, rather than time so the warmup on differently performing hardware or JVM implementations might be doing different things. JMH's purpose is to take that guesswork out of exactly that kind of microbenchmark.
nqzero 5y ago
i've seen similar performance deltas from run to run for both jmh and non-jmh benchmarks, though i don't remember any of them being a 100% delta. very annoying because it means you need to wrap the benchmark in a bash script and average over a dozen+ invocations of java to get semi-meaningful results
cospaia 5y ago
Thanks! I'll check JMH out now. Some week ago, while I thought the problem was todo with something in Clojure - https://twitter.com/pappapez/status/1486339518902984707?s=20&t=lxMxDvX9iNmR75GykyJTPg https://twitter.com/pappapez/status/1486339518902984707?s=20... - I ran async-profiler via clj-async-profiler. It turned out to not be granular enough. OTOH, the problem had different clues then and I should try it again now with the latest Java-only repro.

winrid 5y ago

Would love to see the output of this if someone does it. Sometimes things like this are caused by your code being "un-jitted", in the event some criteria is met, like an exception being thrown for example (although BitSet does not look to do that internally).

pvg 5y ago

This is probably the first thing the author should try. Just a quick look at their own harness, it's sort of trying to guess what 'warmup' is required. The JIT kicks in at invocation counts, among other things, rather than time so the warmup on differently performing hardware or JVM implementations might be doing different things. JMH's purpose is to take that guesswork out of exactly that kind of microbenchmark.

nqzero 5y ago

i've seen similar performance deltas from run to run for both jmh and non-jmh benchmarks, though i don't remember any of them being a 100% delta. very annoying because it means you need to wrap the benchmark in a bash script and average over a dozen+ invocations of java to get semi-meaningful results

cospaia 5y ago

Thanks! I'll check JMH out now. Some week ago, while I thought the problem was todo with something in Clojure - https://twitter.com/pappapez/status/1486339518902984707?s=20&t=lxMxDvX9iNmR75GykyJTPg https://twitter.com/pappapez/status/1486339518902984707?s=20... - I ran async-profiler via clj-async-profiler. It turned out to not be granular enough. OTOH, the problem had different clues then and I should try it again now with the latest Java-only repro.