The new backend is called "bitcask". Here is an introduction to it: http://downloads.basho.com/papers/bitcask-intro.pdf http://downloads.basho.com/papers/bitc

The new backend is called "bitcask". Here is an introduction to it: http://downloads.basho.com/papers/bitcask-intro.pdf http://downloads.basho.com/papers/bitcask-intro.pdf This will be fast indeed. They mention 5000-6000 writes per second which seems very low to me however. Then again, they do not mention the size of the values written. I too became enamored with log-based storage engines in December of last year. I created one myself to play around and learn. I moved on before adding in compaction, but the performance was "excellent" (200K writes per second, 5K random reads per second). Granted, the data model is literally just key-value however. http://github.com/fictorial/logstore http://github.com/fictorial/logstore http://github.com/fictorial/logstore/commit/9ffa46a249a62a78976f457b2de220ef75c98871 http://github.com/fictorial/logstore/commit/9ffa46a249a62a78... Read a paper by Ousterhout about log-based storage here: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.5365 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.... Maybe of interest is a somewhat wacky project in which data is stored in sparse files. The big limitations here are that your system needs support for sparse files, you use a number of pre-created sparse files, and each item's maximum size is fixed. Pretty big limitations! Then again, for trivial purposes, it is ridiculously fast since an item's file and location is trivially computed and I/O uses mmap. http://github.com/fictorial/sparsestore http://github.com/fictorial/sparsestore Sorry, I got excited that there was someone else who was interested in wacky data storage engines!