3 ms·

I'm not sure memory locality effects can be considered secondary, unless their effects are completely dwarfed by something else the app is doing (e.g. there's n

by vitalyd 11y ago

I'm not sure memory locality effects can be considered secondary, unless their effects are completely dwarfed by something else the app is doing (e.g. there's no point in discussing this topic for i/o bound workloads). I don't think malloc/free should enter this conversation because languages that use malloc/free do so very infrequently, and for the cases where dynamic memory needs to be allocated frequently, they use specialized memory managers within the application. This is also subject to which allocator is used and what the application's allocation pattern is. There're suboptimal GC mechanics as well in some cases, such as CMS tenured space using free chunk lists with no compaction, so any young GC that requires promotion can possibly increase the young GC time because the GC needs to find appropriate free block size, and if there's fragmentation, this may take a while. Point being is that malloc/free vs GC isn't quite as clear cut on its own, nevermind that malloc/free aren't called that often. Generally speaking, though, if you can give GC ample headroom in terms of RAM, it'll have better throughput than incessant malloc/free use (which, I argue, is rare in properly written applications).

pron 11y ago
> I'm not sure memory locality effects can be considered secondary What is secondary isn't the TLAB/stack performance ratio, but that ratio vs malloc/TLAB. Also, I'm not sure why you think locality matters much here in the case of stack reuse. Within each frame, the stores always come first, and those go in the store buffer (and the reads are from the store buffer, too), so those are pretty benign cache misses. > which, I argue, is rare in properly written applications Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? Remember, we're not talking about a DSP, a microcontroller or a net router when we're discussing GCs, but big, complex applications. Nobody is claiming you can't beat a GC given enough work (though it's harder the more concurrency is involved). Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC.
- vitalyd 11y ago
  Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more concrete. If not specific name, at least the class of allocator. Most of the common ones you'll find support thread-local allocation buffers, for starters. >Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? >Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC. If we're going to talk about databases, then "well-written" better be one of the top concerns, and team size should be irrelevant to that. And the more mechanically sympathetic of a product you're building (and db's are right up there in pretty much all aspects: cpu, i/o, net, mem, etc), the more you need to have control over those resources. Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? nginx? varnish? And, as I mentioned in the other reply, most of the big data java solutions end up rolling their own off-heap mem management infra using the same techniques as you'd use without GC.
  - pron 11y ago
    > Most of the common ones you'll find support thread-local allocation buffers, for starters. And what about concurrent deallocation? > Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? Not too well (basically lots and lots of locking, much of it is very coarse-grained). Our spatial in-memory Java database (SpaceBase) offers an order-of-magnitude better performance in concurrent usage (and much better scalability with core number). We do over 200K transactions (with lots of contention) per second on a single 4-core laptop without breaking a sweat (concurrently with the application itself), and over a million on a large server (with some careful tuning). But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). In both cases, the amount of effort put into the Java solutions is orders of magnitude less than the C/C++ solutions. > nginx? varnish? Those are (virtually) read-only use cases. That is very easy to do concurrently no matter how you manage concurrency. The trick is concurrent writes, not reads.

by vitalyd 11y ago

pron 11y ago

> I'm not sure memory locality effects can be considered secondary What is secondary isn't the TLAB/stack performance ratio, but that ratio vs malloc/TLAB. Also, I'm not sure why you think locality matters much here in the case of stack reuse. Within each frame, the stores always come first, and those go in the store buffer (and the reads are from the store buffer, too), so those are pretty benign cache misses. > which, I argue, is rare in properly written applications Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? Remember, we're not talking about a DSP, a microcontroller or a net router when we're discussing GCs, but big, complex applications. Nobody is claiming you can't beat a GC given enough work (though it's harder the more concurrency is involved). Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC.

vitalyd 11y ago
Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more concrete. If not specific name, at least the class of allocator. Most of the common ones you'll find support thread-local allocation buffers, for starters. >Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? >Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC. If we're going to talk about databases, then "well-written" better be one of the top concerns, and team size should be irrelevant to that. And the more mechanically sympathetic of a product you're building (and db's are right up there in pretty much all aspects: cpu, i/o, net, mem, etc), the more you need to have control over those resources. Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? nginx? varnish? And, as I mentioned in the other reply, most of the big data java solutions end up rolling their own off-heap mem management infra using the same techniques as you'd use without GC.
- pron 11y ago
  > Most of the common ones you'll find support thread-local allocation buffers, for starters. And what about concurrent deallocation? > Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? Not too well (basically lots and lots of locking, much of it is very coarse-grained). Our spatial in-memory Java database (SpaceBase) offers an order-of-magnitude better performance in concurrent usage (and much better scalability with core number). We do over 200K transactions (with lots of contention) per second on a single 4-core laptop without breaking a sweat (concurrently with the application itself), and over a million on a large server (with some careful tuning). But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). In both cases, the amount of effort put into the Java solutions is orders of magnitude less than the C/C++ solutions. > nginx? varnish? Those are (virtually) read-only use cases. That is very easy to do concurrently no matter how you manage concurrency. The trick is concurrent writes, not reads.