4 ms·

> (OpenJDK's ZGC has typical pause times measured in double/triple-digit microseconds, and the worst case rarely exceeds 1ms for a reasonable allocation rate --

by znpy 3y ago

> (OpenJDK's ZGC has typical pause times measured in double/triple-digit microseconds, and the worst case rarely exceeds 1ms for a reasonable allocation rate -- the pauses are in the same ballpark as OS-induced ones) We've been benchmarking ZGC and Shenandoah at work, and the p100 pause time is usually below 500us (micro-seconds). ZGC seems to be performing a bit better, as it seems to be performing less pauses than Shenandoah (hence doing more work/pause). We still have to run tests-in-production, but so far it seems that GC pauses are largely a solved issue when using ZGC (and Generational ZGC since Java21).

deleted 3y ago
[deleted]
Galanwe 3y ago
> We've been benchmarking ZGC and Shenandoah at work, and the p100 pause time is usually below 500us > so far it seems that GC pauses are largely a solved issue when using ZGC What? 500us is abysmally slow. Most projects I work on have a latency budget of less than 10us, the average being 2us. That is the budget for the whole wire in/wire out for a packet. Even for less latency sensitive workloads, 500us is a no go for most networking application.
- pron 3y ago
  We're talking about occasional hiccups, not an average-case response-latency overhead. You can't get worst-case latency of 2-10us with a non-realtime kernel. Even a page fault could take longer than that.
  - Galanwe 3y ago
    > You can't get worst-case latency of 2-10us with a non-realtime kernel. Even a page fault could take longer than that. You obviously can, and this has nothing to do with the kernel being real-time or not. There is no situation I can think of where a page fault should occur on a properly setup system running a production networking software, meaning no swap, huge TLB, and proper memory management.
    - pron 3y ago
      If you can think of "no situation" where a server may incur a page fault, forced preemption, or need to perform any I/O to a service/database, then I hope you at least recognise that your world in no way represents the state of server software at large because none of these things is true for the vast majority of server software. In a former life I worked on some safety-critical onboard avionics software for an ultrasonic platform, and 2us was around the upper-limit worst-case latency (i.e. you'll kill someone if you miss that deadline); still, it's not the kind of requirements the vast majority of software finds itself under. When working over the internet, some of the very best services are at >10ms ping latency anyway, where a 500us hiccup is imperceptible.
      - Galanwe 3y ago
        > If you can think of "no situation" where a server may incur a page fault, forced preemption, or need to perform any I/O to a service/database, then I hope you at least recognise that your world in no way represents the state of server software at large I won't deny that the majority of software out there is not latency sensitive, but the context of this article is specifically targeting those softwares that are _not_ using garbage collection, arguing that it is undeservedly overlooked. OP even adds that GC is a "solved problem" because some GC implementation is 500us worst case latency. My point is that the article author, and OP, are mistaken. Because if you are in the category of "I write server side software without GC" (e. g. C/C++), then 500us is horribly wrong. Your point being that 500us is fine for most software out there is surely true, but not relevant, because if that is the case, you're probably not using C/C++, thus this article is not targeting you. In _my world_ as you phrase it, traffic is unshaped. You need to be able to support line rate, otherwise packets are dropped, and hell breaks loose.
        naasking 3y ago
        > because if that is the case, you're probably not using C/C++ The point is that this claim is more wrong than it should be, eg. that C/C++ is still used more than it should be partly because these GC myths persist, hence the article.
pron 3y ago
FYI, ZGC doesn't perform any collection work in the the stop-the-world pauses. They are only required to get all mutator threads to atomically observe the increment of the "GC epoch" for all threads. All actual work, both marking and compaction, is done by GC threads running concurrently with the mutator threads or by the mutator threads themselves as they run. It is only when allocation rate is very, very high that an allocating mutator thread will be paused ("throttled") for a significant amount of time to allow the GC to catch up by freeing up memory (and if you hit these cases, then you might be better off using a throughput-oriented collector).
- hawk_ 3y ago
  Strangely in our workloads we have noticed generational ZGC latencies are better than G1 at ~99th-99.9th percentile or below but worse at percentiles above that. The allocation rate is moderate.
  - kaba0 3y ago
    Above that you might easily get measuring artifacts, like the OS swapping out your process once, or so.
    - hawk_ 3y ago
      Yes - but it's quite consistent. If it was 'noise' it wouldn't be.
  - loeg 3y ago
    It can be easy to game metrics like this -- trading off p100 for p99.9 or p99. E.g., defer all the expensive work to 1 in every 10,000 operations.
- znpy 3y ago
  > FYI, ZGC doesn't perform any collection work in the the stop-the-world pauses. Yeah i know, but on a certain level I don’t care what it does or does not do. I care about my application not having latency spikes due to stop the world pauses :)

by znpy 3y ago

deleted 3y ago

[deleted]

Galanwe 3y ago

> We've been benchmarking ZGC and Shenandoah at work, and the p100 pause time is usually below 500us > so far it seems that GC pauses are largely a solved issue when using ZGC What? 500us is abysmally slow. Most projects I work on have a latency budget of less than 10us, the average being 2us. That is the budget for the whole wire in/wire out for a packet. Even for less latency sensitive workloads, 500us is a no go for most networking application.

pron 3y ago
We're talking about occasional hiccups, not an average-case response-latency overhead. You can't get worst-case latency of 2-10us with a non-realtime kernel. Even a page fault could take longer than that.
- Galanwe 3y ago
  > You can't get worst-case latency of 2-10us with a non-realtime kernel. Even a page fault could take longer than that. You obviously can, and this has nothing to do with the kernel being real-time or not. There is no situation I can think of where a page fault should occur on a properly setup system running a production networking software, meaning no swap, huge TLB, and proper memory management.
  - pron 3y ago
    If you can think of "no situation" where a server may incur a page fault, forced preemption, or need to perform any I/O to a service/database, then I hope you at least recognise that your world in no way represents the state of server software at large because none of these things is true for the vast majority of server software. In a former life I worked on some safety-critical onboard avionics software for an ultrasonic platform, and 2us was around the upper-limit worst-case latency (i.e. you'll kill someone if you miss that deadline); still, it's not the kind of requirements the vast majority of software finds itself under. When working over the internet, some of the very best services are at >10ms ping latency anyway, where a 500us hiccup is imperceptible.
    - Galanwe 3y ago
      > If you can think of "no situation" where a server may incur a page fault, forced preemption, or need to perform any I/O to a service/database, then I hope you at least recognise that your world in no way represents the state of server software at large I won't deny that the majority of software out there is not latency sensitive, but the context of this article is specifically targeting those softwares that are _not_ using garbage collection, arguing that it is undeservedly overlooked. OP even adds that GC is a "solved problem" because some GC implementation is 500us worst case latency. My point is that the article author, and OP, are mistaken. Because if you are in the category of "I write server side software without GC" (e. g. C/C++), then 500us is horribly wrong. Your point being that 500us is fine for most software out there is surely true, but not relevant, because if that is the case, you're probably not using C/C++, thus this article is not targeting you. In _my world_ as you phrase it, traffic is unshaped. You need to be able to support line rate, otherwise packets are dropped, and hell breaks loose.
      - naasking 3y ago
        > because if that is the case, you're probably not using C/C++ The point is that this claim is more wrong than it should be, eg. that C/C++ is still used more than it should be partly because these GC myths persist, hence the article.

pron 3y ago

FYI, ZGC doesn't perform any collection work in the the stop-the-world pauses. They are only required to get all mutator threads to atomically observe the increment of the "GC epoch" for all threads. All actual work, both marking and compaction, is done by GC threads running concurrently with the mutator threads or by the mutator threads themselves as they run. It is only when allocation rate is very, very high that an allocating mutator thread will be paused ("throttled") for a significant amount of time to allow the GC to catch up by freeing up memory (and if you hit these cases, then you might be better off using a throughput-oriented collector).

hawk_ 3y ago
Strangely in our workloads we have noticed generational ZGC latencies are better than G1 at ~99th-99.9th percentile or below but worse at percentiles above that. The allocation rate is moderate.
- kaba0 3y ago
  Above that you might easily get measuring artifacts, like the OS swapping out your process once, or so.
  - hawk_ 3y ago
    Yes - but it's quite consistent. If it was 'noise' it wouldn't be.
- loeg 3y ago
  It can be easy to game metrics like this -- trading off p100 for p99.9 or p99. E.g., defer all the expensive work to 1 in every 10,000 operations.
znpy 3y ago
> FYI, ZGC doesn't perform any collection work in the the stop-the-world pauses. Yeah i know, but on a certain level I don’t care what it does or does not do. I care about my application not having latency spikes due to stop the world pauses :)