5 ms·

Great to see the JVM experimenting with low-pause GC. It looks like the maximum pauses of Shenandoah are still well over 1ms, which will still cause a lot of t

Great to see the JVM experimenting with low-pause GC. It looks like the maximum pauses of Shenandoah are still well over 1ms, which will still cause a lot of tail latency in services. (Go reached ~5ms max pauses a few releases ago, but there was still a significant improvement in the behavior of services when pauses were clamped at 100μms in more recent releases.) Definitely the right direction. I hope future versions of Shenandoah will clamp GC pauses even lower.

pron 7y ago
ZGC, the other low latency OpenJDK GC, is now (as of JDK 12) at 1.5ms max pause, 0.5ms average, with a 128GB heap, on SPECjbb2015 (https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Unloading.pdf https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Un... slides 36-40) with throughput that's, as always, much better than Go's. Also, I think Go's pause times are misleading because I believe Go uses throttling and throttling pauses are not counted as GC pauses. One could easily write a GC with absolute 0 max pause: you make each allocation a super-fast, simple pointer bump from a thread local allocation buffer, and when that's out you throttle (i.e. block) the thread forever. Of course the throughput of such a "collector" will end up converging to zero, too. So pause times without throughput numbers are meaningless.
- truth_seeker 7y ago
  Woooooow. This is pretty good news for various big data tools and NoSQL databases which runs on JVM.
- eternalban 7y ago
  If I recall correctly ZGC pause times are O(something x stack-depth) while O(1) memory size.
  - pron 7y ago
    Right. It's linear in the size of the root set, which is #global variables + #references on thread stacks, but constant in the size of the heap.
- shipilev 7y ago
  > throttling pauses are not counted as GC pauses Yes. All GCs I know employ the negative feedback loops between allocators and GCs. Stop-the-world GCs are on the far side of that spectrum: there, allocation failure induces "throttling" that would not unblock the thread until the GC is over. Even contending on shared allocation lock when requesting the new thread-local allocation buffer can be thought as accidental throttling. Concurrent GCs are relying on collecting faster than application allocates, but sometimes application allocates way too much anyway. I believe ZGC does the simplest thing in this case: if allocation fails, the allocation is blocked until memory is available (i.e. until current GC cycle is able to recover some memory). Is this also throttling? I think so. Shenandoah does a few other tricks, including diving into STW when concurrent mode fails. To me, this is also the form of throttling. At the end of the day, I would say that GC pause durations are non-ideal proxies for end-to-end latency. That said, GC pauses do contribute to end-to-end latency very significantly, and so they are not completely meaningless. Their duration should be taken with a fair grain of salt, though, for either Go GC, ZGC, Shenandoah, or whatever else.
  - pron 7y ago
    Oh, my point isn't about whether throttling is good or bad, just about how it's measured. Go's GC's throughput is particularly bad in comparison to OpenJDK's GCs, so talking just about its GC pauses (which do not include throttling, AFAIK) without mentioning throughput gives a false impression.
    - shipilev 7y ago
      I agree with you and present even a stronger statement: talking about GC pauses without measuring the actual end-to-end latencies may give the false impression! The overall throughput might look good, but threads might experience bad local latencies all over the place anyway. In Shenandoah project, that is my recurrent argument for capping the pacing stalls and going to STW mode when bounded pacing did not help.
- pcwalton 7y ago
  Also, Go just doesn't compact the heap, ever. The stop-the-world pauses in the JVM are typically from compaction. It's not really fair to compare a compacting GC to a non-compacting one without acknowledging the huge tradeoff.
  - deleted 7y ago
    [deleted]
  - shipilev 7y ago
    In tracing collectors, marking (and generally walking the heap in possibly random order) would take a significant amount of time. After marking is done, you may decide to move only a few objects, so the overhead of the copying itself is not that large. Updating the references to all those moved objects might take another bulk of time. These stories get better with attempts to segregate the heaps (generational, recording intra-regions references, etc) somewhat. That comes with associated runtime costs to maintain the metadata to support those partial collections, but on the upside it allows to minimize the amount of work done (again), as it only walks/marks copies/updates the sub-heaps. I would not agree with the blanket "The stop-the-world pauses in the JVM are typically from compaction". Concurrent marking is done in CMS, G1, Shenandoah, ZGC. [In first two, there are nitty gritty details about young collections that distort the story] -- and that resolves a significant portion of stop-the-world time. Concurrent copying and updating references is done in Shenandoah, ZGC -- that resolves the rest of it. Of course, you can skip the compact part, and just do a sweep, which frees implementation from dealing with the second part completely. This is not without drawbacks, though: allocation path gets more complicated, fragmentation needs to be dealt with, etc. How far you can get with that, depends on what the use cases really are. As far as I can tell, the damned "CMS concurrent mode failure" caused by heap fragmentation and unwillingness to part with uber-fast bump-ptr allocation paths nudged most JVM people to accept compaction as the go-to answer for reliable GC.
- skybrian 7y ago
  Throughput numbers seem misleading as well for cross-language comparisons? Each language has its own conventions for how much to rely on stacks versus specialized memory pools versus garbage collection. For example, in Go, you often have a stack per request, and can use it for some per-request data as an alternative to GC. That's going to change required throughput for an equivalent amount of work. As will differences in escape analysis. I don't know how much that matters, but it seems like this makes cross-language comparisons pretty subjective. You need to decide what an equivalent app would be in each language. Are you comparing idiomatic or optimized code and what does that really mean? External validity seems pretty limited.
shipilev 7y ago
The pause time behavior depends a lot on what is going on in application itself. Shenandoah wiki has a section on that: https://wiki.openjdk.java.net/display/shenandoah/Main#Main-GeneralIdeas https://wiki.openjdk.java.net/display/shenandoah/Main#Main-G.... It also depends on the hardware, because the same amount of GC work instruction-wise may translate to drastically different cycle/wall-clock time. Try the same 1 GB heap on large x86 desktop and then on Raspberry Pi! This, unfortunately, makes point benchmark numbers not very relevant, and you would want to see how the particular deployment works. With many applications you can push concurrent GCs down to sub-millisecond range without even trying. With some applications, adjusting the application to avoid pitfalls that inflate pauses helps -- the report mentions a few adjustments like that. Edit: Ah, I should have mentioned I am one of those Shenandoah devs, and I have never said "your mileage may vary" that often until diving into GC development work :D
- mcguire 7y ago
  The motto of GC and GC tuning: "This is your mileage in the act of varying."
bradleyjg 7y ago
Depends on what you are trying to do, but I’ll happily trade one to ten millisecond maximum pauses in exchange for better throughput. It’s the hundreds of millisecond pauses that no amount throughput improvements are worth.
lelf 7y ago
Erlang: no global pause at all. Even for major GCs. Basically every Erlang process (≈ “green thread” in common terminology) has its own collector.
- pron 7y ago
  Yeah, but Erlang is very, very (very!) slow. And it, too, has shared data structures like ETS, which aren't so efficiently memory-managed.
  - lelf 7y ago
    [citation needed]. I’d say it’s pretty freaking fast for a bytecode interpreter. Not to mention Hipe.
    - pron 7y ago
      https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ It's pretty bad. We're talking 3-20x slower than Java. FWIW, you can also look at the benchmark shootout game. Erlang/HiPE/BEAM is very cool, but it's just slooooooooow. Now, I'm not trying to say that OpenJDK is "better" than BEAM -- it's a great VM and it's good for lots of stuff -- but comparing performance-related features with Java doesn't make much sense, but it's not really in the same game. Yes, Erlang never has STW pauses, but you'll still get lower latency (and much higher throughput) with OpenJDK and ZGC or Shenandoah, despite the shared heap and occasional (short) STW pauses.
- mcguire 7y ago
  ... which is why Erlang copies the contents of messages between actors. Pony uses a similar heap-per-actor, doesn't copy data, and uses background messages to update GC state between actors. Trade-offs.
- ksec 7y ago
  I assume ( may be wrong ) Project Loom achieves similar benefits when its lands on JVM.
deleted 7y ago
[deleted]

pron 7y ago

ZGC, the other low latency OpenJDK GC, is now (as of JDK 12) at 1.5ms max pause, 0.5ms average, with a 128GB heap, on SPECjbb2015 (https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Unloading.pdf https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Un... slides 36-40) with throughput that's, as always, much better than Go's. Also, I think Go's pause times are misleading because I believe Go uses throttling and throttling pauses are not counted as GC pauses. One could easily write a GC with absolute 0 max pause: you make each allocation a super-fast, simple pointer bump from a thread local allocation buffer, and when that's out you throttle (i.e. block) the thread forever. Of course the throughput of such a "collector" will end up converging to zero, too. So pause times without throughput numbers are meaningless.

truth_seeker 7y ago
Woooooow. This is pretty good news for various big data tools and NoSQL databases which runs on JVM.
eternalban 7y ago
If I recall correctly ZGC pause times are O(something x stack-depth) while O(1) memory size.
- pron 7y ago
  Right. It's linear in the size of the root set, which is #global variables + #references on thread stacks, but constant in the size of the heap.
shipilev 7y ago
> throttling pauses are not counted as GC pauses Yes. All GCs I know employ the negative feedback loops between allocators and GCs. Stop-the-world GCs are on the far side of that spectrum: there, allocation failure induces "throttling" that would not unblock the thread until the GC is over. Even contending on shared allocation lock when requesting the new thread-local allocation buffer can be thought as accidental throttling. Concurrent GCs are relying on collecting faster than application allocates, but sometimes application allocates way too much anyway. I believe ZGC does the simplest thing in this case: if allocation fails, the allocation is blocked until memory is available (i.e. until current GC cycle is able to recover some memory). Is this also throttling? I think so. Shenandoah does a few other tricks, including diving into STW when concurrent mode fails. To me, this is also the form of throttling. At the end of the day, I would say that GC pause durations are non-ideal proxies for end-to-end latency. That said, GC pauses do contribute to end-to-end latency very significantly, and so they are not completely meaningless. Their duration should be taken with a fair grain of salt, though, for either Go GC, ZGC, Shenandoah, or whatever else.
- pron 7y ago
  Oh, my point isn't about whether throttling is good or bad, just about how it's measured. Go's GC's throughput is particularly bad in comparison to OpenJDK's GCs, so talking just about its GC pauses (which do not include throttling, AFAIK) without mentioning throughput gives a false impression.
  - shipilev 7y ago
    I agree with you and present even a stronger statement: talking about GC pauses without measuring the actual end-to-end latencies may give the false impression! The overall throughput might look good, but threads might experience bad local latencies all over the place anyway. In Shenandoah project, that is my recurrent argument for capping the pacing stalls and going to STW mode when bounded pacing did not help.
pcwalton 7y ago
Also, Go just doesn't compact the heap, ever. The stop-the-world pauses in the JVM are typically from compaction. It's not really fair to compare a compacting GC to a non-compacting one without acknowledging the huge tradeoff.
- deleted 7y ago
  [deleted]
- shipilev 7y ago
  In tracing collectors, marking (and generally walking the heap in possibly random order) would take a significant amount of time. After marking is done, you may decide to move only a few objects, so the overhead of the copying itself is not that large. Updating the references to all those moved objects might take another bulk of time. These stories get better with attempts to segregate the heaps (generational, recording intra-regions references, etc) somewhat. That comes with associated runtime costs to maintain the metadata to support those partial collections, but on the upside it allows to minimize the amount of work done (again), as it only walks/marks copies/updates the sub-heaps. I would not agree with the blanket "The stop-the-world pauses in the JVM are typically from compaction". Concurrent marking is done in CMS, G1, Shenandoah, ZGC. [In first two, there are nitty gritty details about young collections that distort the story] -- and that resolves a significant portion of stop-the-world time. Concurrent copying and updating references is done in Shenandoah, ZGC -- that resolves the rest of it. Of course, you can skip the compact part, and just do a sweep, which frees implementation from dealing with the second part completely. This is not without drawbacks, though: allocation path gets more complicated, fragmentation needs to be dealt with, etc. How far you can get with that, depends on what the use cases really are. As far as I can tell, the damned "CMS concurrent mode failure" caused by heap fragmentation and unwillingness to part with uber-fast bump-ptr allocation paths nudged most JVM people to accept compaction as the go-to answer for reliable GC.
skybrian 7y ago
Throughput numbers seem misleading as well for cross-language comparisons? Each language has its own conventions for how much to rely on stacks versus specialized memory pools versus garbage collection. For example, in Go, you often have a stack per request, and can use it for some per-request data as an alternative to GC. That's going to change required throughput for an equivalent amount of work. As will differences in escape analysis. I don't know how much that matters, but it seems like this makes cross-language comparisons pretty subjective. You need to decide what an equivalent app would be in each language. Are you comparing idiomatic or optimized code and what does that really mean? External validity seems pretty limited.

shipilev 7y ago

The pause time behavior depends a lot on what is going on in application itself. Shenandoah wiki has a section on that: https://wiki.openjdk.java.net/display/shenandoah/Main#Main-GeneralIdeas https://wiki.openjdk.java.net/display/shenandoah/Main#Main-G.... It also depends on the hardware, because the same amount of GC work instruction-wise may translate to drastically different cycle/wall-clock time. Try the same 1 GB heap on large x86 desktop and then on Raspberry Pi! This, unfortunately, makes point benchmark numbers not very relevant, and you would want to see how the particular deployment works. With many applications you can push concurrent GCs down to sub-millisecond range without even trying. With some applications, adjusting the application to avoid pitfalls that inflate pauses helps -- the report mentions a few adjustments like that. Edit: Ah, I should have mentioned I am one of those Shenandoah devs, and I have never said "your mileage may vary" that often until diving into GC development work :D

mcguire 7y ago
The motto of GC and GC tuning: "This is your mileage in the act of varying."

bradleyjg 7y ago

Depends on what you are trying to do, but I’ll happily trade one to ten millisecond maximum pauses in exchange for better throughput. It’s the hundreds of millisecond pauses that no amount throughput improvements are worth.

lelf 7y ago

Erlang: no global pause at all. Even for major GCs. Basically every Erlang process (≈ “green thread” in common terminology) has its own collector.

pron 7y ago
Yeah, but Erlang is very, very (very!) slow. And it, too, has shared data structures like ETS, which aren't so efficiently memory-managed.
- lelf 7y ago
  [citation needed]. I’d say it’s pretty freaking fast for a bytecode interpreter. Not to mention Hipe.
  - pron 7y ago
    https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ It's pretty bad. We're talking 3-20x slower than Java. FWIW, you can also look at the benchmark shootout game. Erlang/HiPE/BEAM is very cool, but it's just slooooooooow. Now, I'm not trying to say that OpenJDK is "better" than BEAM -- it's a great VM and it's good for lots of stuff -- but comparing performance-related features with Java doesn't make much sense, but it's not really in the same game. Yes, Erlang never has STW pauses, but you'll still get lower latency (and much higher throughput) with OpenJDK and ZGC or Shenandoah, despite the shared heap and occasional (short) STW pauses.
mcguire 7y ago
... which is why Erlang copies the contents of messages between actors. Pony uses a similar heap-per-actor, doesn't copy data, and uses background messages to update GC state between actors. Trade-offs.
ksec 7y ago
I assume ( may be wrong ) Project Loom achieves similar benefits when its lands on JVM.

deleted 7y ago

[deleted]