3 ms·

ZGC, the other low latency OpenJDK GC, is now (as of JDK 12) at 1.5ms max pause, 0.5ms average, with a 128GB heap, on SPECjbb2015 (https://www.jfokus.se/jfokus1

by pron 8y ago

ZGC, the other low latency OpenJDK GC, is now (as of JDK 12) at 1.5ms max pause, 0.5ms average, with a 128GB heap, on SPECjbb2015 (https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Unloading.pdf https://www.jfokus.se/jfokus19-preso/ZGC-Concurrent-Class-Un... slides 36-40) with throughput that's, as always, much better than Go's. Also, I think Go's pause times are misleading because I believe Go uses throttling and throttling pauses are not counted as GC pauses. One could easily write a GC with absolute 0 max pause: you make each allocation a super-fast, simple pointer bump from a thread local allocation buffer, and when that's out you throttle (i.e. block) the thread forever. Of course the throughput of such a "collector" will end up converging to zero, too. So pause times without throughput numbers are meaningless.

truth_seeker 8y ago
Woooooow. This is pretty good news for various big data tools and NoSQL databases which runs on JVM.
eternalban 8y ago
If I recall correctly ZGC pause times are O(something x stack-depth) while O(1) memory size.
- pron 8y ago
  Right. It's linear in the size of the root set, which is #global variables + #references on thread stacks, but constant in the size of the heap.
shipilev 8y ago
> throttling pauses are not counted as GC pauses Yes. All GCs I know employ the negative feedback loops between allocators and GCs. Stop-the-world GCs are on the far side of that spectrum: there, allocation failure induces "throttling" that would not unblock the thread until the GC is over. Even contending on shared allocation lock when requesting the new thread-local allocation buffer can be thought as accidental throttling. Concurrent GCs are relying on collecting faster than application allocates, but sometimes application allocates way too much anyway. I believe ZGC does the simplest thing in this case: if allocation fails, the allocation is blocked until memory is available (i.e. until current GC cycle is able to recover some memory). Is this also throttling? I think so. Shenandoah does a few other tricks, including diving into STW when concurrent mode fails. To me, this is also the form of throttling. At the end of the day, I would say that GC pause durations are non-ideal proxies for end-to-end latency. That said, GC pauses do contribute to end-to-end latency very significantly, and so they are not completely meaningless. Their duration should be taken with a fair grain of salt, though, for either Go GC, ZGC, Shenandoah, or whatever else.
- pron 8y ago
  Oh, my point isn't about whether throttling is good or bad, just about how it's measured. Go's GC's throughput is particularly bad in comparison to OpenJDK's GCs, so talking just about its GC pauses (which do not include throttling, AFAIK) without mentioning throughput gives a false impression.
  - shipilev 8y ago
    I agree with you and present even a stronger statement: talking about GC pauses without measuring the actual end-to-end latencies may give the false impression! The overall throughput might look good, but threads might experience bad local latencies all over the place anyway. In Shenandoah project, that is my recurrent argument for capping the pacing stalls and going to STW mode when bounded pacing did not help.
pcwalton 8y ago
Also, Go just doesn't compact the heap, ever. The stop-the-world pauses in the JVM are typically from compaction. It's not really fair to compare a compacting GC to a non-compacting one without acknowledging the huge tradeoff.
- deleted 8y ago
  [deleted]
- shipilev 8y ago
  In tracing collectors, marking (and generally walking the heap in possibly random order) would take a significant amount of time. After marking is done, you may decide to move only a few objects, so the overhead of the copying itself is not that large. Updating the references to all those moved objects might take another bulk of time. These stories get better with attempts to segregate the heaps (generational, recording intra-regions references, etc) somewhat. That comes with associated runtime costs to maintain the metadata to support those partial collections, but on the upside it allows to minimize the amount of work done (again), as it only walks/marks copies/updates the sub-heaps. I would not agree with the blanket "The stop-the-world pauses in the JVM are typically from compaction". Concurrent marking is done in CMS, G1, Shenandoah, ZGC. [In first two, there are nitty gritty details about young collections that distort the story] -- and that resolves a significant portion of stop-the-world time. Concurrent copying and updating references is done in Shenandoah, ZGC -- that resolves the rest of it. Of course, you can skip the compact part, and just do a sweep, which frees implementation from dealing with the second part completely. This is not without drawbacks, though: allocation path gets more complicated, fragmentation needs to be dealt with, etc. How far you can get with that, depends on what the use cases really are. As far as I can tell, the damned "CMS concurrent mode failure" caused by heap fragmentation and unwillingness to part with uber-fast bump-ptr allocation paths nudged most JVM people to accept compaction as the go-to answer for reliable GC.
skybrian 8y ago
Throughput numbers seem misleading as well for cross-language comparisons? Each language has its own conventions for how much to rely on stacks versus specialized memory pools versus garbage collection. For example, in Go, you often have a stack per request, and can use it for some per-request data as an alternative to GC. That's going to change required throughput for an equivalent amount of work. As will differences in escape analysis. I don't know how much that matters, but it seems like this makes cross-language comparisons pretty subjective. You need to decide what an equivalent app would be in each language. Are you comparing idiomatic or optimized code and what does that really mean? External validity seems pretty limited.

by pron 8y ago

truth_seeker 8y ago

Woooooow. This is pretty good news for various big data tools and NoSQL databases which runs on JVM.

eternalban 8y ago

If I recall correctly ZGC pause times are O(something x stack-depth) while O(1) memory size.

pron 8y ago
Right. It's linear in the size of the root set, which is #global variables + #references on thread stacks, but constant in the size of the heap.

shipilev 8y ago

> throttling pauses are not counted as GC pauses Yes. All GCs I know employ the negative feedback loops between allocators and GCs. Stop-the-world GCs are on the far side of that spectrum: there, allocation failure induces "throttling" that would not unblock the thread until the GC is over. Even contending on shared allocation lock when requesting the new thread-local allocation buffer can be thought as accidental throttling. Concurrent GCs are relying on collecting faster than application allocates, but sometimes application allocates way too much anyway. I believe ZGC does the simplest thing in this case: if allocation fails, the allocation is blocked until memory is available (i.e. until current GC cycle is able to recover some memory). Is this also throttling? I think so. Shenandoah does a few other tricks, including diving into STW when concurrent mode fails. To me, this is also the form of throttling. At the end of the day, I would say that GC pause durations are non-ideal proxies for end-to-end latency. That said, GC pauses do contribute to end-to-end latency very significantly, and so they are not completely meaningless. Their duration should be taken with a fair grain of salt, though, for either Go GC, ZGC, Shenandoah, or whatever else.

pron 8y ago
Oh, my point isn't about whether throttling is good or bad, just about how it's measured. Go's GC's throughput is particularly bad in comparison to OpenJDK's GCs, so talking just about its GC pauses (which do not include throttling, AFAIK) without mentioning throughput gives a false impression.
- shipilev 8y ago
  I agree with you and present even a stronger statement: talking about GC pauses without measuring the actual end-to-end latencies may give the false impression! The overall throughput might look good, but threads might experience bad local latencies all over the place anyway. In Shenandoah project, that is my recurrent argument for capping the pacing stalls and going to STW mode when bounded pacing did not help.

pcwalton 8y ago

Also, Go just doesn't compact the heap, ever. The stop-the-world pauses in the JVM are typically from compaction. It's not really fair to compare a compacting GC to a non-compacting one without acknowledging the huge tradeoff.

deleted 8y ago
[deleted]
shipilev 8y ago
In tracing collectors, marking (and generally walking the heap in possibly random order) would take a significant amount of time. After marking is done, you may decide to move only a few objects, so the overhead of the copying itself is not that large. Updating the references to all those moved objects might take another bulk of time. These stories get better with attempts to segregate the heaps (generational, recording intra-regions references, etc) somewhat. That comes with associated runtime costs to maintain the metadata to support those partial collections, but on the upside it allows to minimize the amount of work done (again), as it only walks/marks copies/updates the sub-heaps. I would not agree with the blanket "The stop-the-world pauses in the JVM are typically from compaction". Concurrent marking is done in CMS, G1, Shenandoah, ZGC. [In first two, there are nitty gritty details about young collections that distort the story] -- and that resolves a significant portion of stop-the-world time. Concurrent copying and updating references is done in Shenandoah, ZGC -- that resolves the rest of it. Of course, you can skip the compact part, and just do a sweep, which frees implementation from dealing with the second part completely. This is not without drawbacks, though: allocation path gets more complicated, fragmentation needs to be dealt with, etc. How far you can get with that, depends on what the use cases really are. As far as I can tell, the damned "CMS concurrent mode failure" caused by heap fragmentation and unwillingness to part with uber-fast bump-ptr allocation paths nudged most JVM people to accept compaction as the go-to answer for reliable GC.

skybrian 8y ago

Throughput numbers seem misleading as well for cross-language comparisons? Each language has its own conventions for how much to rely on stacks versus specialized memory pools versus garbage collection. For example, in Go, you often have a stack per request, and can use it for some per-request data as an alternative to GC. That's going to change required throughput for an equivalent amount of work. As will differences in escape analysis. I don't know how much that matters, but it seems like this makes cross-language comparisons pretty subjective. You need to decide what an equivalent app would be in each language. Are you comparing idiomatic or optimized code and what does that really mean? External validity seems pretty limited.