7 ms·

I'd recommend a process rather than a handful of implementation-specific rules of thumb that are, IMO, at risk of being applied too generally. I'd hate someone

I'd recommend a process rather than a handful of implementation-specific rules of thumb that are, IMO, at risk of being applied too generally. I'd hate someone to read this article and then go ahead and avoid 'new' wherever possible because they think that's how you make code fast, or avoiding boxing and polymorphic call sites, etc. My process for writing fast code on the JVM: 1) Measure. Set up a benchmark so you know whether you're on the happy path or not, or whether you've fallen off a performance cliff, for whatever reason. Make this part of your testing suite with some kind of notification for regressions. 2) Start small, ideally with a do-nothing loop over the input. This gives you a baseline; you can't go faster than a do-nothing loop (presuming you can't skip part of your input, which is an algorithm problem, not specific to JVM optimization). 3) Incrementally build up your algorithm and pay attention to when it falls off the performance cliff, using your benchmark from (1). If and when you fall off the performance cliff, that's when you start bringing in tricks like avoiding new, ensuring call sites are monomorphic / bimorphic, avoiding boxing, reducing pointer indirections and other cache friendly code, etc. Another trick to consider is playing around with inlining, but not in the way you might think: try pushing infrequently executed code (conditionals) one level deeper in the call stack (i.e. making the body of an if-block into a method and calling it). The idea here is to encourage inlining of method doing the calling. Inlining is where the JVM gets to specialize your code to the specific situation at hand, but the JVM is reluctant to inline big methods because it has a time budget. So you need to help it.

kasey_junk 9y ago
Be careful with 2) because of inlining. Can’t count the number of times I wrote benchmarks that got completely inlined away.
- hyperpape 9y ago
  Agreed. The vast majority of the time you should be using JMH for microbenchmarks on the JVM. If you read the samples and understand them and apply those ideas, it takes care of the inlining problem.
- vvanders 9y ago
  Also for(Foo x : y) will allocate iterators on some JVM implementations which will nail you on the GC.
  - _old_dude_ 9y ago
    It's not on some JVM implementations, to be able to remove the Iterator creation, the VM requires the call to y.iterator() to be either monomorphic or bimorphic. Hence the advice "Keep Things Monomorphic or Bimorphic" !
    - jnordwick 9y ago
      Even if it inline the iteration creation and calls, then hoist and remove the allocation, you're likely to see suboptimal instructions. On hot paths, I often find it better to just avoid iterators in the first place than trying to clean up the mess later.
      - _old_dude_ 9y ago
        yes, the Iterator protocol of Java is sub-optimal - hasNext() has to be called twice so you hope the VM will kick the common sub expression elimination which i have found works well only if you do not do any side effect in hasNext. - and most of the Iterator are fail fast so the code has to check that the modCount of the collection has not changed. but from my own experience, you usually do not have to optimize that far, removing that 'synchronized' block, is far more common.
  - hyperpape 9y ago
    Is this really a consideration outside of cases where someone is tuning their system so that it never does garbage collection?
    - vvanders 9y ago
      Very much so, I see this all the time in production Android code. Someone throws in an iterator like this in an onDraw() and now your super-fast code is hitting a GC pause every hundred ms or so.
- sorokod 9y ago
  If you are not using JMH (http://openjdk.java.net/projects/code-tools/jmh/ http://openjdk.java.net/projects/code-tools/jmh/) for micro-benchmarking, you are very likely doing it wrong. Inlining is just a tip of the gotchas iceberg.
cdman 9y ago
^^^ this. I would also mention the "performance tuning mindmap for Java": https://shipilev.net/talks/devoxx-Nov2012-perfMethodology-mindmap.pdf https://shipilev.net/talks/devoxx-Nov2012-perfMethodology-mi...
kevindication 9y ago
Not really disagreeing with your sentiment, but he laid it out nicely in the "Defining the Hot Path" section. That is honestly the most valuable lesson a newer programmer could take away from this gist.
ccozan 9y ago
Regarding 1) I went down the road of optimization and the code was so fast that we had to switch to nanosec to see something. I was quite impressed what the JiT could do.
jnordwick 9y ago
I'm not really a fan of your "best practices". Java often has unique problems where this generic advice breaks down (I do a lot of low latency and high performance work), and sometimes you need mechanism sympathy instead of relying on tools so much. 1- The author does tell you to understand your hot path. However in Java it is often difficult to pin down GC related issues and just easier to prevent them in the first place. I've tried to go through code bases others have written top clean up allocation problems and they become so embedded in the code that it is easier to just rewrite. Or some issues like finalizer/weak ref issues are very difficult to find in your measurements. 2 and 3- I've never seen this work. This is a great way to get a bunch of faulty analysis or not even know you're going off track to begin with. This article and many of the tips are written when you are trying to be "as fast as possible" and where each microsecond counts. I've seen a lot of projects go down in flames where they treated performance as something secondary the can be written into the code in a second pass. When you ate having to deal with issues like the article describes you should take a more first principles approach and keep your eye on the proper idioms inn the first place.
- barrkel 9y ago
  sometimes you need mechanism sympathy instead of relying on tools so much If you need mechanical sympathy throughout you still need solid measurements, you still need to know when you fall off performance cliffs, but you should be experienced enough not to be misled by articles like this one. That last bit was my biggest concern. Because very very few Java apps have performance bottlenecks throughout. That's more typically seen in game engines, where the whole app is a hot loop, and they're not often written in Java. As to whether it works or not: I've seen it work repeatedly for scanning and parsing. I don't know your situation and can't say why it hasn't worked for you. But again, measurement is the most important bit, and you still need to notice the cliff dropping.
  - jnordwick 9y ago
    This after the fact measurement heavy method of optimizatization method often doesnt work in these systems because it is more often death by a thousand paper cuts than a single line of code. You won't ever notice yourself going off the track even after youre done you likely won't notice. I do a lot of latency and throughout sensitive work, and far more often the bigger problem is losing a few tenths of a percent on each poor decision in your hot path. It becomes very very difficult to fix.
ballenarosada 9y ago
I'm a Java programmer who has tried and failed to do some optimizations this year following your general guidelines. I'm extremely happy to finally see a piece of writing that expands my mental model of what is actually happening on the JVM. If we just keep repeating the same general truisms about measurement and inlining, then nobody will ever advance beyond my own low level of understanding.
slaymaker1907 9y ago
You can tune the maximum size of methods as an argument to the JVM to inline as opposed to manually inlining. Also, I think the first rule of microbenchmarks on the JVM is don't do them. The second rule is to use a proper setup for these benchmarks such as JMH due to the complexities introduced by the JIT and GC.
vardump 9y ago
Some bits and pieces that come to mind: Make use of JVM and CPU performance counters. They're amazing for discovery and understanding of performance matters. Quick googling found these: https://zeroproductionincidents.wordpress.com/tag/jvm-performance-counters/ https://zeroproductionincidents.wordpress.com/tag/jvm-perfor... http://www.peternier.com/projects/overseer/overseer.php http://www.peternier.com/projects/overseer/overseer.php Make sure you also test in actual application context. Microbenchmarking runs into risk of consuming shared CPU resources (such as L1 cache) and exhausting them when running an actual application. In other words, something that is fast in a microbenchmark can perform badly in an actual application. Other than that, one thing no one mentioned is to arrange data in cache-friendly format. Sometimes it might mean ugly things like arrays of primitive types instead of arrays of objects... False sharing is another real performance killer in parallel code. IMHO, writing high performance Java code is a lot harder than high performance C/C++.
- barrkel 9y ago
  > one thing no one mentioned is to arrange data in cache-friendly format I said: reducing pointer indirections and other cache friendly code (just saying!) I agree that it's harder. It's harder because the platform is unpredictable and capricious, it's liable to change from one release to the next, and if you have C-oriented ideas around mechanical sympathy, they can lead you astray. For example, manual loop unrolling is seldom a win in Java (often, the array access checks kill you). What you want is to make your code simple enough for the JVM to unroll it (it'll be able to prove to itself that you don't exceed the bounds and elide checks, especially if you help out with an explicit comparison up front). You're in a dialogue with a reasonably smart compiler, and it's a bit of a guessing game hunting for the happy path. That's why my step 2 and 3 involve incrementally building up your algorithm: you want to stay on the happy path, because if you stray off it, you need to get back on to it again before you change more stuff. Usually the way to go is to write straightforward code (common idioms get more optimization effort from the JVM team) and keep it simple enough to stay in the optimizer time budget (splitting it up where necessary).
  - vardump 9y ago
tehlike 9y ago
Most of performance comes down to profiling. You cannot improve performance without knowing the bottlenecks, and profiling is the best way to get it from real data/usage patterns.

kasey_junk 9y ago

Be careful with 2) because of inlining. Can’t count the number of times I wrote benchmarks that got completely inlined away.

hyperpape 9y ago
Agreed. The vast majority of the time you should be using JMH for microbenchmarks on the JVM. If you read the samples and understand them and apply those ideas, it takes care of the inlining problem.
vvanders 9y ago
Also for(Foo x : y) will allocate iterators on some JVM implementations which will nail you on the GC.
- _old_dude_ 9y ago
  It's not on some JVM implementations, to be able to remove the Iterator creation, the VM requires the call to y.iterator() to be either monomorphic or bimorphic. Hence the advice "Keep Things Monomorphic or Bimorphic" !
  - jnordwick 9y ago
    Even if it inline the iteration creation and calls, then hoist and remove the allocation, you're likely to see suboptimal instructions. On hot paths, I often find it better to just avoid iterators in the first place than trying to clean up the mess later.
    - _old_dude_ 9y ago
      yes, the Iterator protocol of Java is sub-optimal - hasNext() has to be called twice so you hope the VM will kick the common sub expression elimination which i have found works well only if you do not do any side effect in hasNext. - and most of the Iterator are fail fast so the code has to check that the modCount of the collection has not changed. but from my own experience, you usually do not have to optimize that far, removing that 'synchronized' block, is far more common.
- hyperpape 9y ago
  Is this really a consideration outside of cases where someone is tuning their system so that it never does garbage collection?
  - vvanders 9y ago
    Very much so, I see this all the time in production Android code. Someone throws in an iterator like this in an onDraw() and now your super-fast code is hitting a GC pause every hundred ms or so.
sorokod 9y ago
If you are not using JMH (http://openjdk.java.net/projects/code-tools/jmh/ http://openjdk.java.net/projects/code-tools/jmh/) for micro-benchmarking, you are very likely doing it wrong. Inlining is just a tip of the gotchas iceberg.

cdman 9y ago

^^^ this. I would also mention the "performance tuning mindmap for Java": https://shipilev.net/talks/devoxx-Nov2012-perfMethodology-mindmap.pdf https://shipilev.net/talks/devoxx-Nov2012-perfMethodology-mi...

kevindication 9y ago

Not really disagreeing with your sentiment, but he laid it out nicely in the "Defining the Hot Path" section. That is honestly the most valuable lesson a newer programmer could take away from this gist.

ccozan 9y ago

Regarding 1) I went down the road of optimization and the code was so fast that we had to switch to nanosec to see something. I was quite impressed what the JiT could do.

jnordwick 9y ago

I'm not really a fan of your "best practices". Java often has unique problems where this generic advice breaks down (I do a lot of low latency and high performance work), and sometimes you need mechanism sympathy instead of relying on tools so much. 1- The author does tell you to understand your hot path. However in Java it is often difficult to pin down GC related issues and just easier to prevent them in the first place. I've tried to go through code bases others have written top clean up allocation problems and they become so embedded in the code that it is easier to just rewrite. Or some issues like finalizer/weak ref issues are very difficult to find in your measurements. 2 and 3- I've never seen this work. This is a great way to get a bunch of faulty analysis or not even know you're going off track to begin with. This article and many of the tips are written when you are trying to be "as fast as possible" and where each microsecond counts. I've seen a lot of projects go down in flames where they treated performance as something secondary the can be written into the code in a second pass. When you ate having to deal with issues like the article describes you should take a more first principles approach and keep your eye on the proper idioms inn the first place.

barrkel 9y ago
sometimes you need mechanism sympathy instead of relying on tools so much If you need mechanical sympathy throughout you still need solid measurements, you still need to know when you fall off performance cliffs, but you should be experienced enough not to be misled by articles like this one. That last bit was my biggest concern. Because very very few Java apps have performance bottlenecks throughout. That's more typically seen in game engines, where the whole app is a hot loop, and they're not often written in Java. As to whether it works or not: I've seen it work repeatedly for scanning and parsing. I don't know your situation and can't say why it hasn't worked for you. But again, measurement is the most important bit, and you still need to notice the cliff dropping.
- jnordwick 9y ago
  This after the fact measurement heavy method of optimizatization method often doesnt work in these systems because it is more often death by a thousand paper cuts than a single line of code. You won't ever notice yourself going off the track even after youre done you likely won't notice. I do a lot of latency and throughout sensitive work, and far more often the bigger problem is losing a few tenths of a percent on each poor decision in your hot path. It becomes very very difficult to fix.

ballenarosada 9y ago

I'm a Java programmer who has tried and failed to do some optimizations this year following your general guidelines. I'm extremely happy to finally see a piece of writing that expands my mental model of what is actually happening on the JVM. If we just keep repeating the same general truisms about measurement and inlining, then nobody will ever advance beyond my own low level of understanding.

slaymaker1907 9y ago

You can tune the maximum size of methods as an argument to the JVM to inline as opposed to manually inlining. Also, I think the first rule of microbenchmarks on the JVM is don't do them. The second rule is to use a proper setup for these benchmarks such as JMH due to the complexities introduced by the JIT and GC.

vardump 9y ago

Some bits and pieces that come to mind: Make use of JVM and CPU performance counters. They're amazing for discovery and understanding of performance matters. Quick googling found these: https://zeroproductionincidents.wordpress.com/tag/jvm-performance-counters/ https://zeroproductionincidents.wordpress.com/tag/jvm-perfor... http://www.peternier.com/projects/overseer/overseer.php http://www.peternier.com/projects/overseer/overseer.php Make sure you also test in actual application context. Microbenchmarking runs into risk of consuming shared CPU resources (such as L1 cache) and exhausting them when running an actual application. In other words, something that is fast in a microbenchmark can perform badly in an actual application. Other than that, one thing no one mentioned is to arrange data in cache-friendly format. Sometimes it might mean ugly things like arrays of primitive types instead of arrays of objects... False sharing is another real performance killer in parallel code. IMHO, writing high performance Java code is a lot harder than high performance C/C++.

barrkel 9y ago
> one thing no one mentioned is to arrange data in cache-friendly format I said: reducing pointer indirections and other cache friendly code (just saying!) I agree that it's harder. It's harder because the platform is unpredictable and capricious, it's liable to change from one release to the next, and if you have C-oriented ideas around mechanical sympathy, they can lead you astray. For example, manual loop unrolling is seldom a win in Java (often, the array access checks kill you). What you want is to make your code simple enough for the JVM to unroll it (it'll be able to prove to itself that you don't exceed the bounds and elide checks, especially if you help out with an explicit comparison up front). You're in a dialogue with a reasonably smart compiler, and it's a bit of a guessing game hunting for the happy path. That's why my step 2 and 3 involve incrementally building up your algorithm: you want to stay on the happy path, because if you stray off it, you need to get back on to it again before you change more stuff. Usually the way to go is to write straightforward code (common idioms get more optimization effort from the JVM team) and keep it simple enough to stay in the optimizer time budget (splitting it up where necessary).
- vardump 9y ago

tehlike 9y ago

Most of performance comes down to profiling. You cannot improve performance without knowing the bottlenecks, and profiling is the best way to get it from real data/usage patterns.