3 ms·

It's quite common in concurrent services that non-experts write. But the more interesting cases are things like Moka. In a simple evaluation (and, of course, no

by pron 15d ago

It's quite common in concurrent services that non-experts write. But the more interesting cases are things like Moka. In a simple evaluation (and, of course, not much can be extrapolated from any benchmark) Java's old Caffeine library had lower latencies in all percentiles at twice the throughput as Moka (at 90% cache hit rate), as the latter spent 41% of CPU (on top of the cost of malloc/free) on epoch based reclamation.

steveklabnik 15d ago
> and, of course, not much can be extrapolated from any benchmark Right, so one case (which I certainly believe is possible) is very different from “time and time again.”
- pron 15d ago
  By time and time again I meant concurrent services that are written by people who are not experts at low-level programming. The irony is that they don't see "CPU spent on memory management" as they do in Java not because there's less of it - quite often it's much, much more - but because it's simply not measured and reported. As for the caching test, it's just technically interesting, because the JVM was designed to address the performance issues we suffered from in large C++ programs (all the JVM engineers are, of course, C++ people), both due to compilation and to memory management, and we regularly compare both our compilation and memory management algorithms to other approaches, and it just so happens that last week one of our GC engineers compared Caffeine to Moka and saw how CPU-intensive the memory management work is compared to ZGC (he was particularly interested in this because caching is one of the more challenging workloads for generational moving GCs because a cache deals with many old objects, whereas generational GCs tend to focus more on young objects, and he wanted to make sure that our GCs help reduce the high memory-management overheads associated with low-level languages even in this challenging scenario).

by pron 15d ago

steveklabnik 15d ago

> and, of course, not much can be extrapolated from any benchmark Right, so one case (which I certainly believe is possible) is very different from “time and time again.”

pron 15d ago
By time and time again I meant concurrent services that are written by people who are not experts at low-level programming. The irony is that they don't see "CPU spent on memory management" as they do in Java not because there's less of it - quite often it's much, much more - but because it's simply not measured and reported. As for the caching test, it's just technically interesting, because the JVM was designed to address the performance issues we suffered from in large C++ programs (all the JVM engineers are, of course, C++ people), both due to compilation and to memory management, and we regularly compare both our compilation and memory management algorithms to other approaches, and it just so happens that last week one of our GC engineers compared Caffeine to Moka and saw how CPU-intensive the memory management work is compared to ZGC (he was particularly interested in this because caching is one of the more challenging workloads for generational moving GCs because a cache deals with many old objects, whereas generational GCs tend to focus more on young objects, and he wanted to make sure that our GCs help reduce the high memory-management overheads associated with low-level languages even in this challenging scenario).