3 ms·

> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way t

by pron 7y ago

> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size. [1]: https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main

MrBuddyCasino 7y ago
That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.
- naasking 7y ago
  Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.
vbezhenar 7y ago
This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.
- pron 7y ago
  That's a question of throughput, not latency, and there's material about ZGC's throughput as well if you're interested. E.g. Go's GC has good latency but horrible throughput; ZGC is much better.
hedora 7y ago
The Java developers have been promising to deliver a performant GC since the 1.0 days in 1990’s, and they’ve failed to deliver one 14 times in a row. Also, 1ms is an eternity on modern hardware. That’s enough time to handle thousands of disk I/Os or network packets. That means each GC cycle can latency blip 1000’s of other machines.
- pron 7y ago
  > and they’ve failed to deliver one 14 times in a row. Failed at what exactly? People are happy with what he have now, and generally happier with every release (there have been some issues around G1 vs. the now-defunct CMS collector in some special circumstances). Netflix, Amazon, Apple, Google, and Twitter run much or most of their infrastructure on OpenJDK, without even using the newest collectors (AFAIK). > Also, 1ms is an eternity on modern hardware. That depends if you're looking at latency or throughput. Java's GCs have achieved good throughput a long time ago. But ZGC isn't a throughput collector; it is intended for online applications that aren't CPU bound. If you're serving transactions and your application pauses for no more than 1ms every few seconds you're probably OK. There are kernel subsystems that may cause a similar or higher worst-case blip. In any event, Java is not intended for those who must get 100% of the performance the hardware can support, and are willing to pay a high price for getting there, nor is it intended for specialized, niche usage, where perfect performance is easy to achieve. It is intended to get to 95-98% performance at the lowest effort, for a wide variety of applications, and its main runtime cost now is in memory footprint. It's not intended to replace C/C++; we've got Zig for that.
  - EricBurnett 7y ago
    I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.
- MrBuddyCasino 7y ago
  Thats not exactly a fair characterisation. The Java GCs have been state of the art for a long time. Maybe they used to optimise too much for throughput, not latency, but that has changed.
- thu2111 7y ago
  That seems like kind of an extreme position. They've delivered the most performant GCs around, many of which are performant enough for a huge number of users. Also, 1ms is an eternity on modern hardware I hate to break it to you but malloc and free aren't instant either. Also ZGC can do less than 1msec pauses, often way less. At some point it just doesn't matter anymore for virtually all users. That means each GC cycle can latency blip 1000’s of other machines. Linux kernel timeslice is longer than 1 msec, so good luck if your server runs >1 process of any kind at all.

by pron 7y ago

MrBuddyCasino 7y ago

That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.

naasking 7y ago
Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.

vbezhenar 7y ago

This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.

pron 7y ago
That's a question of throughput, not latency, and there's material about ZGC's throughput as well if you're interested. E.g. Go's GC has good latency but horrible throughput; ZGC is much better.

hedora 7y ago

The Java developers have been promising to deliver a performant GC since the 1.0 days in 1990’s, and they’ve failed to deliver one 14 times in a row. Also, 1ms is an eternity on modern hardware. That’s enough time to handle thousands of disk I/Os or network packets. That means each GC cycle can latency blip 1000’s of other machines.

pron 7y ago
> and they’ve failed to deliver one 14 times in a row. Failed at what exactly? People are happy with what he have now, and generally happier with every release (there have been some issues around G1 vs. the now-defunct CMS collector in some special circumstances). Netflix, Amazon, Apple, Google, and Twitter run much or most of their infrastructure on OpenJDK, without even using the newest collectors (AFAIK). > Also, 1ms is an eternity on modern hardware. That depends if you're looking at latency or throughput. Java's GCs have achieved good throughput a long time ago. But ZGC isn't a throughput collector; it is intended for online applications that aren't CPU bound. If you're serving transactions and your application pauses for no more than 1ms every few seconds you're probably OK. There are kernel subsystems that may cause a similar or higher worst-case blip. In any event, Java is not intended for those who must get 100% of the performance the hardware can support, and are willing to pay a high price for getting there, nor is it intended for specialized, niche usage, where perfect performance is easy to achieve. It is intended to get to 95-98% performance at the lowest effort, for a wide variety of applications, and its main runtime cost now is in memory footprint. It's not intended to replace C/C++; we've got Zig for that.
- EricBurnett 7y ago
  I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.
MrBuddyCasino 7y ago
Thats not exactly a fair characterisation. The Java GCs have been state of the art for a long time. Maybe they used to optimise too much for throughput, not latency, but that has changed.
thu2111 7y ago
That seems like kind of an extreme position. They've delivered the most performant GCs around, many of which are performant enough for a huge number of users. Also, 1ms is an eternity on modern hardware I hate to break it to you but malloc and free aren't instant either. Also ZGC can do less than 1msec pauses, often way less. At some point it just doesn't matter anymore for virtually all users. That means each GC cycle can latency blip 1000’s of other machines. Linux kernel timeslice is longer than 1 msec, so good luck if your server runs >1 process of any kind at all.