3 ms·

Could you expand a bit more on the drawbacks of BerkeleyDB JE which you found? Thanks!

• strlen 15y ago
  BerkeleyDB JE is for the most part excellent code (as excepted, given the calibre of the team at Sleepycat/Oracle that built it -- what else should one expect out of a startup founded by Margo Seltzer and Keith Bostic?). That said, the main drawbacks are: * At most, I can give the JVM about 18-22 gb of heap, out of that, at most 8-12gb (usually 10) of heap can go to BerkeleyDB's cache. If I give BDB-Je too little heap, I run the risk of it not having enough space for buffers (for cleaner thread, for scanning through the entries). If I give Bdb too much cache, memory pressure becomes an issue. So now there's 10-12gb of memory (per machine) that's neither use for caching data by BerkeleyDB itself, nor used for the OS page cache. * The index takes up surprisingly much space. In most cases, the index still fits in memory -- but it could definitely be smaller. * Couple of implementation issues that are being addressed by the BDB-Je team and aren't really Java specific. I won't go into them. That said there are many pluses to BerkeleyDB JE: * Log structured B+Tree. Awesome for use with SSDs. SSDs also make the penalty of going outside the in-memory cache much less costly, a random seek for reads is 0.2 ms, and due to log structures design random writes -- not very efficient on SSDs -- don't happen. Additionally, log-structured design means less wear on an SSD. That said, I'd still strongly suggest using an SSD even with a conventional B+Tree (e.g., MySQL InnoDB) -- but benchmark your application first. * Avoids overhead of JNI * As I've mentioned in my comment, being on the JVM means you can spend a lot more time thinking about concurrency. The locking/latching design in BDB JE is very well made.