5 ms·

My favorite database by far today is LMDB (B+Tree).[0] Performance is insane, and very low-variance. Reads scale linearly with core counts, and it has a lot of

My favorite database by far today is LMDB (B+Tree).[0] Performance is insane, and very low-variance. Reads scale linearly with core counts, and it has a lot of useful index types and knobs to get maximum performance. What am I most looking forward to using later this year? ScyllaDB[1] and CockroachDB[2], both in conjunction with LMDB. [0] http://104.237.133.194/doc/ http://104.237.133.194/doc/ [1] http://www.scylladb.com/ http://www.scylladb.com/ [2] https://www.cockroachlabs.com/ https://www.cockroachlabs.com/

filereaper 10y ago
Really liking CockroachDB so far. Also reading through their site, I didn't know Google's Spanner required AtomicClocks! Holy deep pockets Batman!
- jzelinskie 10y ago
  I used to think this as well, until I started talking to more shops that own their own hardware. As it turns out, having atomic clocks isn't quite as extraordinary as you think. If there's enough competition in the public cloud, we might be seeing them there pretty soon.
- technion 10y ago
  Not quite atomic - but an interesting Raspberry Pi project is an NTP server with a GPS unit. It doesn't take deep pockets a all and it offers far more accurate time sync than any average person needs. NTP will rate it at a lower stratum than any network-only time server. I had one in the NTP pool for a while.
- MichaelGG 10y ago
  I don't think such clocks are that expensive. A quick search shows several companies offering them ready to go. I'd guess that even relative to the engineering time involved, the cost of the clocks was irrelevant. GPS NTP servers start at $1000, but I dunno how much that delivers. The actual GPS signal is probably accurate enough, but the rest of the hardware chain on a $1000 box might not be <1ms. Google might have also wanted a backup in case antennas are disturbed or GPS signal somehow interrupted? But even then - just because Google chose 7ms doesn't mean everyone else has to. Drop a GPS NTP master at each site and live with that? I wouldn't imagine it'd get much worse than 10-20ms? I don't really know a whole lot about this though.
- jasonwatkinspdx 10y ago
  Only a small number, used as insurance/backup for problems with GPS. It's not really a huge problem for someone buying many racks of their own equipment.
hendzen 10y ago
If I'm not mistaken, doesn't LMDB have a single, global write lock?
- erichocean 10y ago
  > doesn't LMDB have a single, global write lock? It does, yes. The best architecture when using LMDB is to have a single writer thread, and many independent reader threads. That said, it is nice to be able to run the occasional separate process to do some maintenance task on the database (e.g. to do some writes), and the global lock comes in handy then. The main reason a single write thread design isn't a problem is you can do a million or more separate, fully transactional and isolated write transactions per second on a single core with LMDB. Writes are simply not the bottleneck. You can't get anywhere near that kind of write performance with a fully transactional and concurrent write system like CockroachDB, not even with 100x the hardware. However, some of the eventually consistent databases, like ScyllaDB, can scale up to more total writes per second (but of course you give up the transactions, and the consistency). The actual issue I have with using LMDB in this way is keeping it fed! My code is now written in C++, I'm using Aeron for high-performance messaging and Cap'n Proto for messages (eliminates serialization/deserialization) as well as Cap'n Proto for storing objects inside LMDB. I'm nowhere near LMDB's write thread being the bottleneck, and I'll only get close to it once I'm on a 20Gbit/sec machine. IMO a properly designed system will always and everywhere be bandwidth constrained, not CPU or "architecture" constrained. (Put another way, everyone has access to supercomputers these days, we just use them poorly most of the time.)
  - lj3 10y ago
    Do you have any opinions on Cap'n Proto vs Extprot[0]? [0]: https://github.com/mfp/extprot https://github.com/mfp/extprot
    - erichocean 10y ago
      I haven't seen Extprot before you mentioned it. That said, it appears to require deserialization and that's something we avoid. LMDB can hand you back a pointer to your object in the memory map, and we can easily use that pointer with Cap'n Proto without any calls to malloc().

filereaper 10y ago

Really liking CockroachDB so far. Also reading through their site, I didn't know Google's Spanner required AtomicClocks! Holy deep pockets Batman!

jzelinskie 10y ago
I used to think this as well, until I started talking to more shops that own their own hardware. As it turns out, having atomic clocks isn't quite as extraordinary as you think. If there's enough competition in the public cloud, we might be seeing them there pretty soon.
technion 10y ago
Not quite atomic - but an interesting Raspberry Pi project is an NTP server with a GPS unit. It doesn't take deep pockets a all and it offers far more accurate time sync than any average person needs. NTP will rate it at a lower stratum than any network-only time server. I had one in the NTP pool for a while.
MichaelGG 10y ago
I don't think such clocks are that expensive. A quick search shows several companies offering them ready to go. I'd guess that even relative to the engineering time involved, the cost of the clocks was irrelevant. GPS NTP servers start at $1000, but I dunno how much that delivers. The actual GPS signal is probably accurate enough, but the rest of the hardware chain on a $1000 box might not be <1ms. Google might have also wanted a backup in case antennas are disturbed or GPS signal somehow interrupted? But even then - just because Google chose 7ms doesn't mean everyone else has to. Drop a GPS NTP master at each site and live with that? I wouldn't imagine it'd get much worse than 10-20ms? I don't really know a whole lot about this though.
jasonwatkinspdx 10y ago
Only a small number, used as insurance/backup for problems with GPS. It's not really a huge problem for someone buying many racks of their own equipment.

hendzen 10y ago

If I'm not mistaken, doesn't LMDB have a single, global write lock?

erichocean 10y ago
> doesn't LMDB have a single, global write lock? It does, yes. The best architecture when using LMDB is to have a single writer thread, and many independent reader threads. That said, it is nice to be able to run the occasional separate process to do some maintenance task on the database (e.g. to do some writes), and the global lock comes in handy then. The main reason a single write thread design isn't a problem is you can do a million or more separate, fully transactional and isolated write transactions per second on a single core with LMDB. Writes are simply not the bottleneck. You can't get anywhere near that kind of write performance with a fully transactional and concurrent write system like CockroachDB, not even with 100x the hardware. However, some of the eventually consistent databases, like ScyllaDB, can scale up to more total writes per second (but of course you give up the transactions, and the consistency). The actual issue I have with using LMDB in this way is keeping it fed! My code is now written in C++, I'm using Aeron for high-performance messaging and Cap'n Proto for messages (eliminates serialization/deserialization) as well as Cap'n Proto for storing objects inside LMDB. I'm nowhere near LMDB's write thread being the bottleneck, and I'll only get close to it once I'm on a 20Gbit/sec machine. IMO a properly designed system will always and everywhere be bandwidth constrained, not CPU or "architecture" constrained. (Put another way, everyone has access to supercomputers these days, we just use them poorly most of the time.)
- lj3 10y ago
  Do you have any opinions on Cap'n Proto vs Extprot[0]? [0]: https://github.com/mfp/extprot https://github.com/mfp/extprot
  - erichocean 10y ago
    I haven't seen Extprot before you mentioned it. That said, it appears to require deserialization and that's something we avoid. LMDB can hand you back a pointer to your object in the memory map, and we can easily use that pointer with Cap'n Proto without any calls to malloc().