3 ms·

Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more

by vitalyd 11y ago

Ok, we keep talking about malloc -- which malloc impl are you specifically referring to? There are many allocators out there these days, so let's be a bit more concrete. If not specific name, at least the class of allocator. Most of the common ones you'll find support thread-local allocation buffers, for starters. >Sure, it is rare in "well written applications", but how costly is it to write a well-written application in a large team, and how much extra performance can you get? >Also, think about what kind of data we're talking about. The interesting data is database data, and that has both arbitrary lifetime as well as concurrent read/write. And if you don't use malloc/free, at best you need to write your own manual memory allocator which is just as complex, and at worst you basically need to write your own GC. If we're going to talk about databases, then "well-written" better be one of the top concerns, and team size should be irrelevant to that. And the more mechanically sympathetic of a product you're building (and db's are right up there in pretty much all aspects: cpu, i/o, net, mem, etc), the more you need to have control over those resources. Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? nginx? varnish? And, as I mentioned in the other reply, most of the big data java solutions end up rolling their own off-heap mem management infra using the same techniques as you'd use without GC.

pron 11y ago
> Most of the common ones you'll find support thread-local allocation buffers, for starters. And what about concurrent deallocation? > Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? Not too well (basically lots and lots of locking, much of it is very coarse-grained). Our spatial in-memory Java database (SpaceBase) offers an order-of-magnitude better performance in concurrent usage (and much better scalability with core number). We do over 200K transactions (with lots of contention) per second on a single 4-core laptop without breaking a sweat (concurrently with the application itself), and over a million on a large server (with some careful tuning). But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). In both cases, the amount of effort put into the Java solutions is orders of magnitude less than the C/C++ solutions. > nginx? varnish? Those are (virtually) read-only use cases. That is very easy to do concurrently no matter how you manage concurrency. The trick is concurrent writes, not reads.
- anarazel 11y ago
  > But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). It's hardly surprising that a vendor's own benchmark shows it as winning against the competition.
  - pron 11y ago
    Let's suppose for a second that the numbers are biased in the Java databases' favor. I doubt that other benchmarks would show such dramatically different results that would have those databases losing by much. So maybe they're not ahead as those number show (though I have little reason to doubt them), but a little behind. The difference would still not justify the claim that C++ provides superior performance for databases. Also, it's an open source database with no commercial support by the authors, so I wouldn't really call them "vendors".
  - pron 11y ago
    Let's suppose for a second that the numbers are biased in the Java databases' favor. I doubt that other benchmarks would show such dramatically different results that would have those databases losing by much. So maybe they're not ahead as those number show (though I have little reason to doubt them), but a little behind. The difference would still not justify the claim that C++ provides superior performance for databases. Also, it's an open source database with no commercial support by the authors, so I wouldn't really call them "vendors".

by vitalyd 11y ago

pron 11y ago

> Most of the common ones you'll find support thread-local allocation buffers, for starters. And what about concurrent deallocation? > Have you, for example, looked at how postgresql manages memory? sqlite? redis? memcached? Not too well (basically lots and lots of locking, much of it is very coarse-grained). Our spatial in-memory Java database (SpaceBase) offers an order-of-magnitude better performance in concurrent usage (and much better scalability with core number). We do over 200K transactions (with lots of contention) per second on a single 4-core laptop without breaking a sweat (concurrently with the application itself), and over a million on a large server (with some careful tuning). But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). In both cases, the amount of effort put into the Java solutions is orders of magnitude less than the C/C++ solutions. > nginx? varnish? Those are (virtually) read-only use cases. That is very easy to do concurrently no matter how you manage concurrency. The trick is concurrent writes, not reads.

anarazel 11y ago
> But even for less super-concurrent databases, C++ databases don't outperform Java ones. In this benchmark, the Java databases (H2 and HSQLDB) almost always outperform MySQL and Postgres: http://www.h2database.com/html/performance.html http://www.h2database.com/html/performance.html (and I don't even know how the Java solutions handle concurrency, whether they do locking, optimistic locking or a clever combination, like SpaceBase). It's hardly surprising that a vendor's own benchmark shows it as winning against the competition.
- pron 11y ago
  Let's suppose for a second that the numbers are biased in the Java databases' favor. I doubt that other benchmarks would show such dramatically different results that would have those databases losing by much. So maybe they're not ahead as those number show (though I have little reason to doubt them), but a little behind. The difference would still not justify the claim that C++ provides superior performance for databases. Also, it's an open source database with no commercial support by the authors, so I wouldn't really call them "vendors".
- pron 11y ago
  Let's suppose for a second that the numbers are biased in the Java databases' favor. I doubt that other benchmarks would show such dramatically different results that would have those databases losing by much. So maybe they're not ahead as those number show (though I have little reason to doubt them), but a little behind. The difference would still not justify the claim that C++ provides superior performance for databases. Also, it's an open source database with no commercial support by the authors, so I wouldn't really call them "vendors".