3 ms·

Thanks for the story details, quite interesting. In the end this was unfortunately a case of picking the wrong tool for the job. Don’t use JITed / GC languages

Thanks for the story details, quite interesting. In the end this was unfortunately a case of picking the wrong tool for the job. Don’t use JITed / GC languages when you‘ve got hard realtime requirements. Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra). Don’t rely on auto-vectorisation in your inner loop if possible, one tiny change could bring that house of cards crashing down. I‘d be interested in how your team ended up picking that tech stack. Was it a „rational“ weighing of options with pros and cons? Was it „eh it‘ll be alright“? Was it personal preference and/or prior experience?

pron 7y ago
> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size. [1]: https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main
- MrBuddyCasino 7y ago
  That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.
  - naasking 7y ago
    Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.
- vbezhenar 7y ago
  This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.
  - pron 7y ago
    That's a question of throughput, not latency, and there's material about ZGC's throughput as well if you're interested. E.g. Go's GC has good latency but horrible throughput; ZGC is much better.
- hedora 7y ago
  The Java developers have been promising to deliver a performant GC since the 1.0 days in 1990’s, and they’ve failed to deliver one 14 times in a row. Also, 1ms is an eternity on modern hardware. That’s enough time to handle thousands of disk I/Os or network packets. That means each GC cycle can latency blip 1000’s of other machines.
  - pron 7y ago
    > and they’ve failed to deliver one 14 times in a row. Failed at what exactly? People are happy with what he have now, and generally happier with every release (there have been some issues around G1 vs. the now-defunct CMS collector in some special circumstances). Netflix, Amazon, Apple, Google, and Twitter run much or most of their infrastructure on OpenJDK, without even using the newest collectors (AFAIK). > Also, 1ms is an eternity on modern hardware. That depends if you're looking at latency or throughput. Java's GCs have achieved good throughput a long time ago. But ZGC isn't a throughput collector; it is intended for online applications that aren't CPU bound. If you're serving transactions and your application pauses for no more than 1ms every few seconds you're probably OK. There are kernel subsystems that may cause a similar or higher worst-case blip. In any event, Java is not intended for those who must get 100% of the performance the hardware can support, and are willing to pay a high price for getting there, nor is it intended for specialized, niche usage, where perfect performance is easy to achieve. It is intended to get to 95-98% performance at the lowest effort, for a wide variety of applications, and its main runtime cost now is in memory footprint. It's not intended to replace C/C++; we've got Zig for that.
    - EricBurnett 7y ago
      I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.
  - MrBuddyCasino 7y ago
    Thats not exactly a fair characterisation. The Java GCs have been state of the art for a long time. Maybe they used to optimise too much for throughput, not latency, but that has changed.

pron 7y ago

> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra) OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size. [1]: https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main

MrBuddyCasino 7y ago
That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.
- naasking 7y ago
  Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.
vbezhenar 7y ago
This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.
- pron 7y ago
  That's a question of throughput, not latency, and there's material about ZGC's throughput as well if you're interested. E.g. Go's GC has good latency but horrible throughput; ZGC is much better.
hedora 7y ago
The Java developers have been promising to deliver a performant GC since the 1.0 days in 1990’s, and they’ve failed to deliver one 14 times in a row. Also, 1ms is an eternity on modern hardware. That’s enough time to handle thousands of disk I/Os or network packets. That means each GC cycle can latency blip 1000’s of other machines.
- pron 7y ago
  > and they’ve failed to deliver one 14 times in a row. Failed at what exactly? People are happy with what he have now, and generally happier with every release (there have been some issues around G1 vs. the now-defunct CMS collector in some special circumstances). Netflix, Amazon, Apple, Google, and Twitter run much or most of their infrastructure on OpenJDK, without even using the newest collectors (AFAIK). > Also, 1ms is an eternity on modern hardware. That depends if you're looking at latency or throughput. Java's GCs have achieved good throughput a long time ago. But ZGC isn't a throughput collector; it is intended for online applications that aren't CPU bound. If you're serving transactions and your application pauses for no more than 1ms every few seconds you're probably OK. There are kernel subsystems that may cause a similar or higher worst-case blip. In any event, Java is not intended for those who must get 100% of the performance the hardware can support, and are willing to pay a high price for getting there, nor is it intended for specialized, niche usage, where perfect performance is easy to achieve. It is intended to get to 95-98% performance at the lowest effort, for a wide variety of applications, and its main runtime cost now is in memory footprint. It's not intended to replace C/C++; we've got Zig for that.
  - EricBurnett 7y ago
    I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.
- MrBuddyCasino 7y ago
  Thats not exactly a fair characterisation. The Java GCs have been state of the art for a long time. Maybe they used to optimise too much for throughput, not latency, but that has changed.