4 ms·

Some bits and pieces that come to mind: Make use of JVM and CPU performance counters. They're amazing for discovery and understanding of performance matters.

Some bits and pieces that come to mind: Make use of JVM and CPU performance counters. They're amazing for discovery and understanding of performance matters. Quick googling found these: https://zeroproductionincidents.wordpress.com/tag/jvm-performance-counters/ https://zeroproductionincidents.wordpress.com/tag/jvm-perfor... http://www.peternier.com/projects/overseer/overseer.php http://www.peternier.com/projects/overseer/overseer.php Make sure you also test in actual application context. Microbenchmarking runs into risk of consuming shared CPU resources (such as L1 cache) and exhausting them when running an actual application. In other words, something that is fast in a microbenchmark can perform badly in an actual application. Other than that, one thing no one mentioned is to arrange data in cache-friendly format. Sometimes it might mean ugly things like arrays of primitive types instead of arrays of objects... False sharing is another real performance killer in parallel code. IMHO, writing high performance Java code is a lot harder than high performance C/C++.

barrkel 9y ago
> one thing no one mentioned is to arrange data in cache-friendly format I said: reducing pointer indirections and other cache friendly code (just saying!) I agree that it's harder. It's harder because the platform is unpredictable and capricious, it's liable to change from one release to the next, and if you have C-oriented ideas around mechanical sympathy, they can lead you astray. For example, manual loop unrolling is seldom a win in Java (often, the array access checks kill you). What you want is to make your code simple enough for the JVM to unroll it (it'll be able to prove to itself that you don't exceed the bounds and elide checks, especially if you help out with an explicit comparison up front). You're in a dialogue with a reasonably smart compiler, and it's a bit of a guessing game hunting for the happy path. That's why my step 2 and 3 involve incrementally building up your algorithm: you want to stay on the happy path, because if you stray off it, you need to get back on to it again before you change more stuff. Usually the way to go is to write straightforward code (common idioms get more optimization effort from the JVM team) and keep it simple enough to stay in the optimizer time budget (splitting it up where necessary).
- vardump 9y ago
  > For example, manual loop unrolling is seldom a win in Java Just like practically everywhere else. Besides, it increases code size, and can decrease performance just for that reason, L1 code cache spills. Intel CPUs have increasingly improved loop buffers as well. If the loop qualifies, unrolling has no positive effect at all. Low end microcontrollers are another matter, that's where unrolling can still sometimes yield significant wins. I think the biggest issue I had with high performance Java is lack of memory alignment control and (at least last time I checked) vector intrinsics. JNI time...
  - physguy1123 9y ago
    I reliably see gains from unrolling relatively simple loops on very recent cpus. Going to 8+ iterations is usually overkill, but it can help. Sometimes one can split a dependency chain by unrolling as well.
    - vardump 9y ago
      > I reliably see gains from unrolling relatively simple loops on very recent cpus. Maybe you didn't have multiple of 4 uops in the loop? Or hit some other (?) loop buffer limitation. Dependency chain splitting is also indeed a good reason to "unroll". Although it's a bit more complex transformation than simple unrolling.
  - barrkel 9y ago
    Loop unrolling can definitely still be a win if it's in the form of vectorising or otherwise processing multiple elements at a time.
username223 9y ago
> IMHO, writing high performance Java code is a lot harder than high performance C/C++. I haven't had to write high performance (numerical) code in Java, but my instinct is to doubt that it's even possible. There are too many levels of abstraction and complexity, which are constantly changing. Does some JIT heuristic get triggered to specialize a method? Does array bounds-check elimination work in this particular case? Does the compiler allocate this array so that SIMD instructions will work? Will the next minor version bump change everything? With C/C++/Fortran you can look at the generated assembly and either beat the compiler into submission, or write the asm yourself. With Java, I don't see a viable path to success. This is Lisp's "sufficiently smart compiler" problem with a 2000s makeover.
- _old_dude_ 9y ago
  Using Java for numerical performance seems not a good idea too me to, the Java JITs seem to general for my taste. Anyway, this is how the Vectorization works in Hotspot: http://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorization_in_HotSpot_JVM.pdf http://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorizatio...
- peoplewindow 9y ago
  C/C++ developers rely on knowledge of compiler optimisations all the time, the most famous case being the NRV optimization in C++. But exploiting knowledge of where data structures will be allocated is another example. Java can be used for numerical work but it's not ideal for the reasons you cite. However this isn't fundamental to the technology or language. It's more a problem of incentives. Java isn't used for such work, so the applications that people care about and try to optimise at the JVM level don't tend to benefit from them, so the cost/benefit ratio of such optimisations seem low, so focus goes elsewhere. C2 can do quite sophisticated auto-vectorisations but I suspect those optimisations rarely trigger because work that benefits from vectorisation was moved into C or assembly a long time ago. It's rather forward looking work. Graal doesn't even focus on it. They put all their effort into optimising large 'business java' type apps, along with language interpreters (Graal's party trick is turning interpreters into fast JIT compilers).

barrkel 9y ago

> one thing no one mentioned is to arrange data in cache-friendly format I said: reducing pointer indirections and other cache friendly code (just saying!) I agree that it's harder. It's harder because the platform is unpredictable and capricious, it's liable to change from one release to the next, and if you have C-oriented ideas around mechanical sympathy, they can lead you astray. For example, manual loop unrolling is seldom a win in Java (often, the array access checks kill you). What you want is to make your code simple enough for the JVM to unroll it (it'll be able to prove to itself that you don't exceed the bounds and elide checks, especially if you help out with an explicit comparison up front). You're in a dialogue with a reasonably smart compiler, and it's a bit of a guessing game hunting for the happy path. That's why my step 2 and 3 involve incrementally building up your algorithm: you want to stay on the happy path, because if you stray off it, you need to get back on to it again before you change more stuff. Usually the way to go is to write straightforward code (common idioms get more optimization effort from the JVM team) and keep it simple enough to stay in the optimizer time budget (splitting it up where necessary).

vardump 9y ago
> For example, manual loop unrolling is seldom a win in Java Just like practically everywhere else. Besides, it increases code size, and can decrease performance just for that reason, L1 code cache spills. Intel CPUs have increasingly improved loop buffers as well. If the loop qualifies, unrolling has no positive effect at all. Low end microcontrollers are another matter, that's where unrolling can still sometimes yield significant wins. I think the biggest issue I had with high performance Java is lack of memory alignment control and (at least last time I checked) vector intrinsics. JNI time...
- physguy1123 9y ago
  I reliably see gains from unrolling relatively simple loops on very recent cpus. Going to 8+ iterations is usually overkill, but it can help. Sometimes one can split a dependency chain by unrolling as well.
  - vardump 9y ago
    > I reliably see gains from unrolling relatively simple loops on very recent cpus. Maybe you didn't have multiple of 4 uops in the loop? Or hit some other (?) loop buffer limitation. Dependency chain splitting is also indeed a good reason to "unroll". Although it's a bit more complex transformation than simple unrolling.
- barrkel 9y ago
  Loop unrolling can definitely still be a win if it's in the form of vectorising or otherwise processing multiple elements at a time.

username223 9y ago

> IMHO, writing high performance Java code is a lot harder than high performance C/C++. I haven't had to write high performance (numerical) code in Java, but my instinct is to doubt that it's even possible. There are too many levels of abstraction and complexity, which are constantly changing. Does some JIT heuristic get triggered to specialize a method? Does array bounds-check elimination work in this particular case? Does the compiler allocate this array so that SIMD instructions will work? Will the next minor version bump change everything? With C/C++/Fortran you can look at the generated assembly and either beat the compiler into submission, or write the asm yourself. With Java, I don't see a viable path to success. This is Lisp's "sufficiently smart compiler" problem with a 2000s makeover.

_old_dude_ 9y ago
Using Java for numerical performance seems not a good idea too me to, the Java JITs seem to general for my taste. Anyway, this is how the Vectorization works in Hotspot: http://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorization_in_HotSpot_JVM.pdf http://cr.openjdk.java.net/~vlivanov/talks/2017_Vectorizatio...
peoplewindow 9y ago
C/C++ developers rely on knowledge of compiler optimisations all the time, the most famous case being the NRV optimization in C++. But exploiting knowledge of where data structures will be allocated is another example. Java can be used for numerical work but it's not ideal for the reasons you cite. However this isn't fundamental to the technology or language. It's more a problem of incentives. Java isn't used for such work, so the applications that people care about and try to optimise at the JVM level don't tend to benefit from them, so the cost/benefit ratio of such optimisations seem low, so focus goes elsewhere. C2 can do quite sophisticated auto-vectorisations but I suspect those optimisations rarely trigger because work that benefits from vectorisation was moved into C or assembly a long time ago. It's rather forward looking work. Graal doesn't even focus on it. They put all their effort into optimising large 'business java' type apps, along with language interpreters (Graal's party trick is turning interpreters into fast JIT compilers).