25 ms·

Shenandoah in OpenJDK 17: Sub-millisecond GC pauses

226 points by BenoitP 5y ago

stonemetal12 5y ago
So given that the JDK is open source and dotnet is open source, would it be possible to port Shenandoah? It is my understanding that D's garbage collector isn't that good, could they pick up an OpenJDK collector?
- kaba0 5y ago
  I am unfortunately not an OpenJDK dev (but would really like to become one down the road), but afaik a GC is way too dependent on the internals of the runtime. The JVM has a very different object structure. Also, I think D allows for pointers which is a no-go with compacting GCs (as those move objects around). So all in all, the algorithm used itself can be ported, but it has many runtime specific logic, which is just as important for great performance.
dig1 5y ago
Awesome work! In our shop, we managed to solve significant application bottlenecks simply by switching to Shenandoah GC. The application in question is very complex, so any kind of rewrite in another language is out of the question. This is the power of JVM we do not get from other languages/VMs. Kudos to JVM (previous and current) architects, developers, and everyone involved with it.
- dis-sys 5y ago
  > This is the power of JVM we do not get from other languages/VMs. sub-millisecond gc pause has been available to Golang users for ages.
  - awild 5y ago
    I'm guessing the parent is referring to the ability to transparently change runtime parameters and benefit from technical improvements without having to switch languages or even needing recompilation. That's not something golang can do given its statically linked afaik
    - dig1 5y ago
      Exactly!
  - kaba0 5y ago
    Only by not allocating garbage in the first place by using “value types”. No way its GC would even get close to the JVM’s (and certain problems do require garbage collection, where the JVM will have better throughput)
- bjarneh 5y ago
  > The application in question is very complex, so any kind of rewrite in another language is out of the question. I think I know what you mean. If a couple of guys quit, the company is in major trouble? :-) Forgot this: /s
  - brianmcc 5y ago
    Not the OP but you can have a large, complex system that's widely understood and supported by a decent sized team, where a rewrite is still not a commercially wise choice
    - dig1 5y ago
      True!
    - jacquesm 5y ago
      This is probably the norm.
    - FieryTransition 5y ago
      Yeah, rewriting everything is almost never a good choice from a business value perspective. Usually, the most sensible thing to do, is to break the system up in components, and rewrite them slowly while keeping interop with the old system. Downtime is no money time. And new features sometimes give more value, even if the programmer is grumpy.
  - deleted 5y ago
    [deleted]
mnd999 5y ago
Everyone else: pauses are bad, we can’t have garbage collection. Java: pauses are bad, let’s make the garbage collector better. Much credit to the Java community for ignoring the noise and building something great that lets most applications have the best of both worlds.
- Semaphor 5y ago
  .NET is also regularly improving their GC, aren’t they?
  - DougBTX 5y ago
    Also notably Go, though really all GC languages which care about performance, of which there are quite a few.
    - the8472 5y ago
      Go isn't playing in the same league though since their GC is non-generational and non-compacting. A lot of the difficulties implementing concurrent GCs comes from moving things between generations and compacting.
      - DougBTX 5y ago
        I had this benchmark[0] in mind while writing the comment, though perhaps Go does well there by avoiding GC rather than having fast GC as-such? That would explain why its performance is close to the non-GC languages. [0] https://awesomeopensource.com/project/ixy-languages/ixy-languages https://awesomeopensource.com/project/ixy-languages/ixy-lang...
        throwaway894345 5y ago
        Without digging into that benchmark, Go leans on stack allocation a lot more than many GC languages, and that’s a big part of why Go is able to achieve otherwise impressive performance. The same mechanism (idiomatic value types) that allows Go to elide GC in many cases also permits better cache coherency—there are fewer pointers into random regions of the heap so cache lines need to be flushed less frequently.
emmericp 5y ago
We wrote some experimental network drivers in high-level languages a few years ago and Java performed better than expected. Yeah, that's a very special and somewhat odd use case, but it was fun :) We've got some graphs [0] comparing throughput and long-tail latency for the various Java GCs available in OpenJDK 12. Shenandoah's worst case pause time was ~45µs which is the same as disabling the GC (Epsilon GC) which is pretty impressive. Overall performance did suffer under Shenandoah a little bit back then, though. However, I've heard that this improved recently. [0] https://github.com/ixy-languages/ixy-languages/blob/master/Java-garbage-collectors.md https://github.com/ixy-languages/ixy-languages/blob/master/J...
- jacquesm 5y ago
  Java can be amazingly performant. One particular example is a video compressor handling many concurrent streams, recompressing them on the fly and shipping them out again saturating a gigabit link.
deleted 5y ago
[deleted]
grogers 5y ago
Shenandoah is keeping pace nicely with ZGC, especially important to make this into the jdk 17 release since that will be what most users target for the next few years. Given the similar feature sets, does anyone with experience running both ZGC and shenandoah have any differentiators?
- rkennke 5y ago
  The main difference from user perspective are 1. ZGC doesn't support compressed references (-XX:+UseCompressedOops) which could affect workloads <32GB heap and 2. Shenandoah GC is not available in Oracle builds
enduku 5y ago
One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further: - Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase - Shenandoah likes ReentrantLocks better than synchronised classes the latter bloats up its monitors and increases the size of the rootset - The reflection problem may initial Shenandoah to run GC cycles even when there is no memory pressure and thus may reduce performance over time. It is better to limit it using reflection inflation flag - On Amazon ECS, provide the CPU share count explicitly; Shenandoah likes more threads
- tpxl 5y ago
  > One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further Other GCs will do the same for you (G1, ZGC) which you can then put into a tool like gceasy to pretty graphs.
- raducu 5y ago
  > - Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase I'm in way disputing this, but why would a weak reference be different than any other reference, beyond the added flexibility it gives the GC (you may or may not GC this reference if you please)? I admit I don't give GC much thought beyond every coupe of years when I switch jobs :)
  - eropple 5y ago
    I haven't looked at Shenandoah and my GC experience is a bit stale, but to determine if something is weakly referenced, you need to determine whether or not something is universally weakly referenced. A strong reference anywhere means it isn't weakly referenced at all, and as I recall it's been thought (for now) that while this may not require stopping the world to sweep the full set of references, it's a safer and more correct option.
uvesten 5y ago
I’m slightly confused as to what the relation is between OpenJDK and Oracle JDK. Are they based on the same codebase? I was under the (apparently mistaken) impression that OpenJDK and the ”official” (Oracle) JDK were totally different implementations, but seeing how Oracle’s JDK just hit 17, and apparently OpenJDK did at the same time I guess they are derived from the same source. Could anyone enlighten me?
- Reason077 5y ago
  They are built from the same code base. Any differences are just down to the license, packaging, etc.
  - acdha 5y ago
    Doesn’t the Oracle one include some relatively obscure but non-free libraries?
    - shellac 5y ago
      It certainly has done, but currently the differences are pretty tiny: https://www.oracle.com/java/technologies/javase/17all-relnotes.html https://www.oracle.com/java/technologies/javase/17all-relnot... It's really just 'usage logging', which is for Oracle's management tools. Their stated aim is to eliminate the functional differences, so presumably they are awaiting a non-proprietary replacement for that feature in JDK.
  - uvesten 5y ago
    Thanks! So, given the recent license changes to Oracle’s distribution, why would anyone use it? ”Support”?
thinkharderdev 5y ago
Great article. The JVM is one of the most astonishing pieces of software ever created. I was looking at gRPC benchmarks the other day (https://github.com/LesnyRumcajs/grpc_bench/wiki/2021-08-30-bench-results https://github.com/LesnyRumcajs/grpc_bench/wiki/2021-08-30-b...) and 6 of the top 7 performing gRPC implementations on 3 core CPUS were on the JVM.
- srazzaque 5y ago
  Completely agree. Say what you will about Java (the language), the JVM is a seriously good piece of kit.
  - bognition 5y ago
    Java is a fantastic language. It's simple and in the right environment can be elegant. It's incredibly easy to ramp up new engineers on it and the tooling thats available make discovery and navigation a breeze. It may lack a lot of the fancier features of some newer languages but its still a very effective language.
    - cygned 5y ago
      > It may lack a lot of the fancier features Coming from JavaScript, TypeScript, Python and Go back to Java I can tell you that’s actually a good thing.
      - bognition 5y ago
        Agree 100%. Let me build stuff using simple easy to understand tools. Cleverness in code starts working against you as a code base and organizations scale.
        throwaway894345 5y ago
        Doesn’t Java tend to have a lot of cleverness/magic? Pretty crazy runtime codegen, reflection, etc? Spring and other DI frameworks seem overly clever to me, as do many ORMs. I guess my standard for a simple, minimal-magic language is Go.
ludamad 5y ago
I saw this start during my internship at Red Hat in 2012 :) cool to see it kicking around
anderson1993 5y ago
Hi. Can the Shenandoah's concurrent stack scanning in RedHat builds of the JDK 17 be considered production ready?
- JanecekPetr 5y ago
  If it's not behind XX:+UnlockExperimentalVMOptions (or a preview/incubator module), it's considered stable and prod-ready, yes. Will there be fixes and improvements? Yes. Should you be afraid to use it for business? No, but obviously test it first.
- the-alchemist 5y ago
  You can buy support from Redhat, which would presumably include support for their JDK build with Shenandoah. Not sure what that means in practice, but if someone is putting money on the line, it must be worth _something_.
ceronman 5y ago
It took me some time working with a non-GC language to fully appreciate how amazing Garbage Collection is. It really makes a lot of things easier, simpler and specially safer. Yes, of course it has some drawbacks. Yes, of course, not every single kind of program can afford to have a GC. But hell, life is so much easier with a GC! I'm very happy to see that there is active research and development happening to make the drawbacks of GC as minimal as possible!
- mikepurvis 5y ago
  I feel like the key is knowing that your problem domain is fine with there always being garbage collection in the loop. Otherwise there's the risk that down the road you'll have to start tracing the code, rewriting things to reuse objects, etc etc.
  - Koshkin 5y ago
    > reuse objects Caching is often not such a bad idea regardless of GC.
    - mikepurvis 5y ago
      Fair, but I think some GC'd languages (Java, JS) end up with a culture of boxed-everything where it's extremely easy to inadvertently throw off a ton of garbage from an inner loop, even doing operations that in other languages would only ever instantiate objects on the stack.
      - tadfisher 5y ago
        Newer Java runtimes are very conscious of tight garbage creation loops, and will generally either unbox or perform some internal object pooling to avoid GC pressure. Value types (targeting JDK 18? or maybe already in 17) will give developers a tool to create new stack-allocated types for performance-critical sections as well
        5y ago
juddgaddie 5y ago
It would have been valuable to see the percentile distribution of latency, 99th, 99.999... max etc
yutijke 5y ago
I am a layman with regards to JVM internals. Are there any differences in the targeted workloads or capabilities between ZGC and Shenandoah GC? Both seem to be marketed for large heaps and low pause times.
- rkennke 5y ago
  The main difference from user perspective are 1. ZGC doesn't support compressed references (-XX:+UseCompressedOops) which could affect workloads <32GB heap and 2. Shenandoah GC is not available in Oracle builds
bob1029 5y ago
The quality of the JVM GC is impressive to me as a .NET developer. I would love to have reliable sub-ms pauses in my apps. I am working on one right now where even small pauses >1ms are potentially detectable by the user.
- kllrnohj 5y ago
  As a Java developer I'd love to have .NET's value types and runtime generics. Something something grass is always greener on the side :)

226 points by BenoitP 5y ago

So given that the JDK is open source and dotnet is open source, would it be possible to port Shenandoah? It is my understanding that D's garbage collector isn't that good, could they pick up an OpenJDK collector?

kaba0 5y ago
I am unfortunately not an OpenJDK dev (but would really like to become one down the road), but afaik a GC is way too dependent on the internals of the runtime. The JVM has a very different object structure. Also, I think D allows for pointers which is a no-go with compacting GCs (as those move objects around). So all in all, the algorithm used itself can be ported, but it has many runtime specific logic, which is just as important for great performance.

dig1 5y ago

Awesome work! In our shop, we managed to solve significant application bottlenecks simply by switching to Shenandoah GC. The application in question is very complex, so any kind of rewrite in another language is out of the question. This is the power of JVM we do not get from other languages/VMs. Kudos to JVM (previous and current) architects, developers, and everyone involved with it.

dis-sys 5y ago
> This is the power of JVM we do not get from other languages/VMs. sub-millisecond gc pause has been available to Golang users for ages.
- awild 5y ago
  I'm guessing the parent is referring to the ability to transparently change runtime parameters and benefit from technical improvements without having to switch languages or even needing recompilation. That's not something golang can do given its statically linked afaik
  - dig1 5y ago
    Exactly!
- kaba0 5y ago
  Only by not allocating garbage in the first place by using “value types”. No way its GC would even get close to the JVM’s (and certain problems do require garbage collection, where the JVM will have better throughput)
bjarneh 5y ago
> The application in question is very complex, so any kind of rewrite in another language is out of the question. I think I know what you mean. If a couple of guys quit, the company is in major trouble? :-) Forgot this: /s
- brianmcc 5y ago
  Not the OP but you can have a large, complex system that's widely understood and supported by a decent sized team, where a rewrite is still not a commercially wise choice
  - dig1 5y ago
    True!
  - jacquesm 5y ago
    This is probably the norm.
  - FieryTransition 5y ago
    Yeah, rewriting everything is almost never a good choice from a business value perspective. Usually, the most sensible thing to do, is to break the system up in components, and rewrite them slowly while keeping interop with the old system. Downtime is no money time. And new features sometimes give more value, even if the programmer is grumpy.
- deleted 5y ago
  [deleted]

mnd999 5y ago

Everyone else: pauses are bad, we can’t have garbage collection. Java: pauses are bad, let’s make the garbage collector better. Much credit to the Java community for ignoring the noise and building something great that lets most applications have the best of both worlds.

Semaphor 5y ago
.NET is also regularly improving their GC, aren’t they?
- DougBTX 5y ago
  Also notably Go, though really all GC languages which care about performance, of which there are quite a few.
  - the8472 5y ago
    Go isn't playing in the same league though since their GC is non-generational and non-compacting. A lot of the difficulties implementing concurrent GCs comes from moving things between generations and compacting.
    - DougBTX 5y ago
      I had this benchmark[0] in mind while writing the comment, though perhaps Go does well there by avoiding GC rather than having fast GC as-such? That would explain why its performance is close to the non-GC languages. [0] https://awesomeopensource.com/project/ixy-languages/ixy-languages https://awesomeopensource.com/project/ixy-languages/ixy-lang...
      - throwaway894345 5y ago
        Without digging into that benchmark, Go leans on stack allocation a lot more than many GC languages, and that’s a big part of why Go is able to achieve otherwise impressive performance. The same mechanism (idiomatic value types) that allows Go to elide GC in many cases also permits better cache coherency—there are fewer pointers into random regions of the heap so cache lines need to be flushed less frequently.

emmericp 5y ago

We wrote some experimental network drivers in high-level languages a few years ago and Java performed better than expected. Yeah, that's a very special and somewhat odd use case, but it was fun :) We've got some graphs [0] comparing throughput and long-tail latency for the various Java GCs available in OpenJDK 12. Shenandoah's worst case pause time was ~45µs which is the same as disabling the GC (Epsilon GC) which is pretty impressive. Overall performance did suffer under Shenandoah a little bit back then, though. However, I've heard that this improved recently. [0] https://github.com/ixy-languages/ixy-languages/blob/master/Java-garbage-collectors.md https://github.com/ixy-languages/ixy-languages/blob/master/J...

jacquesm 5y ago
Java can be amazingly performant. One particular example is a video compressor handling many concurrent streams, recompressing them on the fly and shipping them out again saturating a gigabit link.

deleted 5y ago

[deleted]

grogers 5y ago

Shenandoah is keeping pace nicely with ZGC, especially important to make this into the jdk 17 release since that will be what most users target for the next few years. Given the similar feature sets, does anyone with experience running both ZGC and shenandoah have any differentiators?

rkennke 5y ago
The main difference from user perspective are 1. ZGC doesn't support compressed references (-XX:+UseCompressedOops) which could affect workloads <32GB heap and 2. Shenandoah GC is not available in Oracle builds

enduku 5y ago

One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further: - Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase - Shenandoah likes ReentrantLocks better than synchronised classes the latter bloats up its monitors and increases the size of the rootset - The reflection problem may initial Shenandoah to run GC cycles even when there is no memory pressure and thus may reduce performance over time. It is better to limit it using reflection inflation flag - On Amazon ECS, provide the CPU share count explicitly; Shenandoah likes more threads

tpxl 5y ago
> One neat feature of Shenandoah is once the application shuts down, it prints into the GC log a table depicting how much time each phase of the GC cycle took. This data can be crucial to optimize the GC pause time further Other GCs will do the same for you (G1, ZGC) which you can then put into a tool like gceasy to pretty graphs.
raducu 5y ago
> - Removal of weak references (finalizer, phantom, soft) can help reduce the GC pause time further as these warrant stop the world phase I'm in way disputing this, but why would a weak reference be different than any other reference, beyond the added flexibility it gives the GC (you may or may not GC this reference if you please)? I admit I don't give GC much thought beyond every coupe of years when I switch jobs :)
- eropple 5y ago
  I haven't looked at Shenandoah and my GC experience is a bit stale, but to determine if something is weakly referenced, you need to determine whether or not something is universally weakly referenced. A strong reference anywhere means it isn't weakly referenced at all, and as I recall it's been thought (for now) that while this may not require stopping the world to sweep the full set of references, it's a safer and more correct option.

uvesten 5y ago

I’m slightly confused as to what the relation is between OpenJDK and Oracle JDK. Are they based on the same codebase? I was under the (apparently mistaken) impression that OpenJDK and the ”official” (Oracle) JDK were totally different implementations, but seeing how Oracle’s JDK just hit 17, and apparently OpenJDK did at the same time I guess they are derived from the same source. Could anyone enlighten me?

Reason077 5y ago
They are built from the same code base. Any differences are just down to the license, packaging, etc.
- acdha 5y ago
  Doesn’t the Oracle one include some relatively obscure but non-free libraries?
  - shellac 5y ago
    It certainly has done, but currently the differences are pretty tiny: https://www.oracle.com/java/technologies/javase/17all-relnotes.html https://www.oracle.com/java/technologies/javase/17all-relnot... It's really just 'usage logging', which is for Oracle's management tools. Their stated aim is to eliminate the functional differences, so presumably they are awaiting a non-proprietary replacement for that feature in JDK.
- uvesten 5y ago
  Thanks! So, given the recent license changes to Oracle’s distribution, why would anyone use it? ”Support”?

thinkharderdev 5y ago

Great article. The JVM is one of the most astonishing pieces of software ever created. I was looking at gRPC benchmarks the other day (https://github.com/LesnyRumcajs/grpc_bench/wiki/2021-08-30-bench-results https://github.com/LesnyRumcajs/grpc_bench/wiki/2021-08-30-b...) and 6 of the top 7 performing gRPC implementations on 3 core CPUS were on the JVM.

srazzaque 5y ago
Completely agree. Say what you will about Java (the language), the JVM is a seriously good piece of kit.
- bognition 5y ago
  Java is a fantastic language. It's simple and in the right environment can be elegant. It's incredibly easy to ramp up new engineers on it and the tooling thats available make discovery and navigation a breeze. It may lack a lot of the fancier features of some newer languages but its still a very effective language.
  - cygned 5y ago
    > It may lack a lot of the fancier features Coming from JavaScript, TypeScript, Python and Go back to Java I can tell you that’s actually a good thing.
    - bognition 5y ago
      Agree 100%. Let me build stuff using simple easy to understand tools. Cleverness in code starts working against you as a code base and organizations scale.
      - throwaway894345 5y ago
        Doesn’t Java tend to have a lot of cleverness/magic? Pretty crazy runtime codegen, reflection, etc? Spring and other DI frameworks seem overly clever to me, as do many ORMs. I guess my standard for a simple, minimal-magic language is Go.

ludamad 5y ago

I saw this start during my internship at Red Hat in 2012 :) cool to see it kicking around

anderson1993 5y ago

Hi. Can the Shenandoah's concurrent stack scanning in RedHat builds of the JDK 17 be considered production ready?

JanecekPetr 5y ago
If it's not behind XX:+UnlockExperimentalVMOptions (or a preview/incubator module), it's considered stable and prod-ready, yes. Will there be fixes and improvements? Yes. Should you be afraid to use it for business? No, but obviously test it first.
the-alchemist 5y ago
You can buy support from Redhat, which would presumably include support for their JDK build with Shenandoah. Not sure what that means in practice, but if someone is putting money on the line, it must be worth _something_.

ceronman 5y ago

It took me some time working with a non-GC language to fully appreciate how amazing Garbage Collection is. It really makes a lot of things easier, simpler and specially safer. Yes, of course it has some drawbacks. Yes, of course, not every single kind of program can afford to have a GC. But hell, life is so much easier with a GC! I'm very happy to see that there is active research and development happening to make the drawbacks of GC as minimal as possible!

mikepurvis 5y ago
I feel like the key is knowing that your problem domain is fine with there always being garbage collection in the loop. Otherwise there's the risk that down the road you'll have to start tracing the code, rewriting things to reuse objects, etc etc.
- Koshkin 5y ago
  > reuse objects Caching is often not such a bad idea regardless of GC.
  - mikepurvis 5y ago
    Fair, but I think some GC'd languages (Java, JS) end up with a culture of boxed-everything where it's extremely easy to inadvertently throw off a ton of garbage from an inner loop, even doing operations that in other languages would only ever instantiate objects on the stack.
    - tadfisher 5y ago
      Newer Java runtimes are very conscious of tight garbage creation loops, and will generally either unbox or perform some internal object pooling to avoid GC pressure. Value types (targeting JDK 18? or maybe already in 17) will give developers a tool to create new stack-allocated types for performance-critical sections as well
      - 5y ago

juddgaddie 5y ago

It would have been valuable to see the percentile distribution of latency, 99th, 99.999... max etc

yutijke 5y ago

I am a layman with regards to JVM internals. Are there any differences in the targeted workloads or capabilities between ZGC and Shenandoah GC? Both seem to be marketed for large heaps and low pause times.

rkennke 5y ago
The main difference from user perspective are 1. ZGC doesn't support compressed references (-XX:+UseCompressedOops) which could affect workloads <32GB heap and 2. Shenandoah GC is not available in Oracle builds

bob1029 5y ago

The quality of the JVM GC is impressive to me as a .NET developer. I would love to have reliable sub-ms pauses in my apps. I am working on one right now where even small pauses >1ms are potentially detectable by the user.

kllrnohj 5y ago
As a Java developer I'd love to have .NET's value types and runtime generics. Something something grass is always greener on the side :)