4 ms·

We were surprised by Java's poor performance because the HotSpot JVM also uses a concurrent mark-and-sweep collector, and is famed for the amount of engineering

We were surprised by Java's poor performance because the HotSpot JVM also uses a concurrent mark-and-sweep collector, and is famed for the amount of engineering effort that has gone into it. We suspect our benchmark could be improved by someone with more high-performance JVM experience. [Edit] jcipar on Reddit made some promising progress by tuning the JVM GC parameters for our benchmark: https://www.reddit.com/r/programming/comments/5fyhjb/golangs_realtime_gc_in_theory_and_practice_pusher/daof47v/ https://www.reddit.com/r/programming/comments/5fyhjb/golangs...

fnord123 10y ago
Isn't GC parameter tuning equivalent tuning model parameters and hence susceptible to over fitting? As soon a you patch your service, the parameters may need to change.
- Someone 10y ago
  Not only GC parameter tuning, _all_ parameter tuning. In many cases, technologies have matured enough so that you can build a mental model of them that fits reality both now and in a couple of years time, but that is never guaranteed. For example, in the small: - the performance of a bit shift used to be dependent on the number of bits shifted (https://wiki.neogeodev.org/index.php?title=68k_instructions_timings#Shift_and_rotate_instructions https://wiki.neogeodev.org/index.php?title=68k_instructions_...). Timing of integer and floating point multiplication was dependent on argument values. - on PowerPC, for at least some CPUs, it could be worthwhile to use floating point variables to iterate over an integer range because that kept the integer pipeline free for use doing the actual work. Change the CPU, and you have to change the type of your loop variable. In the large: - the optimal code for lots of data-crunching code is hugely dependent on cache size, sizes and relative speeds of caches, main memory, and disk. - if you swap in another C library, performance (for example of scanf or transcendental functions) can be hugely different. - upgrading the OS may significantly change the relative timings of thread and process creation, changing the best way to solve your problem from multi-process to multi-threaded or vice versa. So yes, GC parameter tuning is a black art that, ideally, eventually should go away, but the difference with other technologies is only gradually, and, at least, you can tune it without recompiling your code. I also think GC is getting mature enough for fairly reliable mental models to form. The throughput/latency tradeoff that this article mentions is an important aspect; minimizing total memory usage may be another.
  - pjmlp 10y ago
    Another example, the pile of switches usually available in C and C++ compilers.
pcwalton 10y ago
The HotSpot GC does compaction, while Go's doesn't IIRC. Compaction is the most difficult part to make concurrent. If you're just measuring pause times, then HotSpot will be at a disadvantage, as it's doing more.
- vvanders 10y ago
  Wait, Go doesn't compact? Is it safe to assume that they have to deal with fragmentation or does Go solve that another way?
  - nickcw 10y ago
    I believe that fragmentation isn't such a problem in Go programs because they make less use of the heap than Java programs. I can't remember where I read this though. I don't know much about the internals of Java, but I know that Go will stack allocate any kind of object if the compiler can prove it doesn't escape, and that in Go lots of things (not just native types) are passed by value on the stack instead of by reference to a heap value.
    - vvanders 10y ago
      Sure, I guess I'm just floored that you'd have a system that doesn't compact and has no manual mechanisms for allocation. I guess it's just a different class of problems that I'm used to dealing with. Fragmentation is something I've seen many times and it usually likes to rear it's head at the worst possible moment.
      - krakensden 10y ago
        > has no manual mechanisms for allocation Golang supports value types for objects, so arrays (or something higher level like sync.Pool) work great for doing arena allocation.
    - pcwalton 10y ago
      > I don't know much about the internals of Java, but I know that Go will stack allocate any kind of object if the compiler can prove it doesn't escape This is also the case in HotSpot.
user5994461 10y ago
Here's a simple practical guide to tune the Java GC properly: http://stackoverflow.com/a/37448602/5994461 http://stackoverflow.com/a/37448602/5994461 The only thing you need to do nowadays is set "-Xmx__g -Xms__g" to configure the min and max heap size (replace __with a number). There is only one rule to follow: The min and max heap must be the EXACT SAME number. (Basically, the heap must be fixed size). If they're not, the GC will resize the heap during the program execution, which is a very expensive process that will freeze the application and ruin performances. --- https://www.reddit.com/r/programming/comments/5fyhjb/golangs_realtime_gc_in_theory_and_practice_pusher/daof47v/ https://www.reddit.com/r/programming/comments/5fyhjb/golangs... That comment is terrible advice. "-Xmx800m" then "-Xmx400m". He sets the max heap without setting the min, plus he runs the test for only a very short time not enough to stabilize the heap nor the optimizer. Bad bad bad.
- titzer 10y ago
  > There is only one rule to follow: The min and max heap must be the EXACT SAME number. (Basically, the heap must be fixed size). If they're not, the GC will resize the heap during the program execution, which is a very expensive process that will freeze the application and ruin performances. I doubt this is true of the JVM GC. Adaptive heap size tuning is also in V8 and it doesn't require an expensive operation like a full GC to accomplish (e.g. the whole heap is not copied during this operation). The GC can simply add more pages of memory after a minor GC and promote some of the objects from the young gen to the old gen. So it doesn't have to be more expensive than a young gen (minor) GC.
  - user5994461 10y ago
    It is true. One of the reddit comment states that they divided the latency by 5 after they reconfigured the min and max heap properly and let the program run for longer. The GC is in a constant battle to keep the program running within memory bounds. If you make it think that the program can run in 20MB by not giving a setting, it will try to fit the program in 20MB and clean the heap all the time (till it gives up and grow the heap). The GC needs to know it's bound to it doesn't bother running when not necessary AND it can decide how much space is available to keep short/long lived objects.
sandGorgon 10y ago
This was a super insightful comment The Java GC does not optimise for low latency exclusively, it tries to strike a balance between many different factors which your analysis entirely ignores! In particular it is compacting (Go's GC is not), which is very useful for long term stability as you can't get deaths due to heap fragmentation, and it tries to collect large amounts of garbage quickly, which Go's GC doesn't try to do (it's pure mark/sweep, not generational), and it tries to be configurable, which Go's GC doesn't care about. For many apps you do tend to want these things, they aren't pointless. This is especially true because for most servers a 100msec pause time is just fine. 8msec is the kind of thing you need if you're doing a 60fps video game but for most servers it's overkill and they'd prefer to bank the performance. To put this in perspective Google is having problems migrating to G1 because even though it gives lower and more predictable pause latencies, it slows their servers down by 10%. A 10% throughput loss is unacceptable at their scale (translates to a 10% increase in java server costs) and they want G1 to become even more configurable to let them pick faster execution at the cost of longer pauses
- ngrilly 10y ago
  > This is especially true because for most servers a 100msec pause time is just fine. I disagree (a bit). A 100 msec pause is perceptible by a human, and can lead a user to leave your website or you app, especially is the pauses are compounded among several serialized remote procedure calls.

fnord123 10y ago

Isn't GC parameter tuning equivalent tuning model parameters and hence susceptible to over fitting? As soon a you patch your service, the parameters may need to change.

Someone 10y ago
Not only GC parameter tuning, _all_ parameter tuning. In many cases, technologies have matured enough so that you can build a mental model of them that fits reality both now and in a couple of years time, but that is never guaranteed. For example, in the small: - the performance of a bit shift used to be dependent on the number of bits shifted (https://wiki.neogeodev.org/index.php?title=68k_instructions_timings#Shift_and_rotate_instructions https://wiki.neogeodev.org/index.php?title=68k_instructions_...). Timing of integer and floating point multiplication was dependent on argument values. - on PowerPC, for at least some CPUs, it could be worthwhile to use floating point variables to iterate over an integer range because that kept the integer pipeline free for use doing the actual work. Change the CPU, and you have to change the type of your loop variable. In the large: - the optimal code for lots of data-crunching code is hugely dependent on cache size, sizes and relative speeds of caches, main memory, and disk. - if you swap in another C library, performance (for example of scanf or transcendental functions) can be hugely different. - upgrading the OS may significantly change the relative timings of thread and process creation, changing the best way to solve your problem from multi-process to multi-threaded or vice versa. So yes, GC parameter tuning is a black art that, ideally, eventually should go away, but the difference with other technologies is only gradually, and, at least, you can tune it without recompiling your code. I also think GC is getting mature enough for fairly reliable mental models to form. The throughput/latency tradeoff that this article mentions is an important aspect; minimizing total memory usage may be another.
- pjmlp 10y ago
  Another example, the pile of switches usually available in C and C++ compilers.

pcwalton 10y ago

The HotSpot GC does compaction, while Go's doesn't IIRC. Compaction is the most difficult part to make concurrent. If you're just measuring pause times, then HotSpot will be at a disadvantage, as it's doing more.

vvanders 10y ago
Wait, Go doesn't compact? Is it safe to assume that they have to deal with fragmentation or does Go solve that another way?
- nickcw 10y ago
  I believe that fragmentation isn't such a problem in Go programs because they make less use of the heap than Java programs. I can't remember where I read this though. I don't know much about the internals of Java, but I know that Go will stack allocate any kind of object if the compiler can prove it doesn't escape, and that in Go lots of things (not just native types) are passed by value on the stack instead of by reference to a heap value.
  - vvanders 10y ago
    Sure, I guess I'm just floored that you'd have a system that doesn't compact and has no manual mechanisms for allocation. I guess it's just a different class of problems that I'm used to dealing with. Fragmentation is something I've seen many times and it usually likes to rear it's head at the worst possible moment.
    - krakensden 10y ago
      > has no manual mechanisms for allocation Golang supports value types for objects, so arrays (or something higher level like sync.Pool) work great for doing arena allocation.
  - pcwalton 10y ago
    > I don't know much about the internals of Java, but I know that Go will stack allocate any kind of object if the compiler can prove it doesn't escape This is also the case in HotSpot.

user5994461 10y ago

Here's a simple practical guide to tune the Java GC properly: http://stackoverflow.com/a/37448602/5994461 http://stackoverflow.com/a/37448602/5994461 The only thing you need to do nowadays is set "-Xmx__g -Xms__g" to configure the min and max heap size (replace __with a number). There is only one rule to follow: The min and max heap must be the EXACT SAME number. (Basically, the heap must be fixed size). If they're not, the GC will resize the heap during the program execution, which is a very expensive process that will freeze the application and ruin performances. --- https://www.reddit.com/r/programming/comments/5fyhjb/golangs_realtime_gc_in_theory_and_practice_pusher/daof47v/ https://www.reddit.com/r/programming/comments/5fyhjb/golangs... That comment is terrible advice. "-Xmx800m" then "-Xmx400m". He sets the max heap without setting the min, plus he runs the test for only a very short time not enough to stabilize the heap nor the optimizer. Bad bad bad.

titzer 10y ago
> There is only one rule to follow: The min and max heap must be the EXACT SAME number. (Basically, the heap must be fixed size). If they're not, the GC will resize the heap during the program execution, which is a very expensive process that will freeze the application and ruin performances. I doubt this is true of the JVM GC. Adaptive heap size tuning is also in V8 and it doesn't require an expensive operation like a full GC to accomplish (e.g. the whole heap is not copied during this operation). The GC can simply add more pages of memory after a minor GC and promote some of the objects from the young gen to the old gen. So it doesn't have to be more expensive than a young gen (minor) GC.
- user5994461 10y ago
  It is true. One of the reddit comment states that they divided the latency by 5 after they reconfigured the min and max heap properly and let the program run for longer. The GC is in a constant battle to keep the program running within memory bounds. If you make it think that the program can run in 20MB by not giving a setting, it will try to fit the program in 20MB and clean the heap all the time (till it gives up and grow the heap). The GC needs to know it's bound to it doesn't bother running when not necessary AND it can decide how much space is available to keep short/long lived objects.

sandGorgon 10y ago

This was a super insightful comment The Java GC does not optimise for low latency exclusively, it tries to strike a balance between many different factors which your analysis entirely ignores! In particular it is compacting (Go's GC is not), which is very useful for long term stability as you can't get deaths due to heap fragmentation, and it tries to collect large amounts of garbage quickly, which Go's GC doesn't try to do (it's pure mark/sweep, not generational), and it tries to be configurable, which Go's GC doesn't care about. For many apps you do tend to want these things, they aren't pointless. This is especially true because for most servers a 100msec pause time is just fine. 8msec is the kind of thing you need if you're doing a 60fps video game but for most servers it's overkill and they'd prefer to bank the performance. To put this in perspective Google is having problems migrating to G1 because even though it gives lower and more predictable pause latencies, it slows their servers down by 10%. A 10% throughput loss is unacceptable at their scale (translates to a 10% increase in java server costs) and they want G1 to become even more configurable to let them pick faster execution at the cost of longer pauses

ngrilly 10y ago
> This is especially true because for most servers a 100msec pause time is just fine. I disagree (a bit). A 100 msec pause is perceptible by a human, and can lead a user to leave your website or you app, especially is the pauses are compounded among several serialized remote procedure calls.