5 ms·

Here is a link to some more 'thorough' benchmarks that compare WiredTiger to many other database back-ends. http://symas.com/mdb/inmem/ http://symas.com/mdb/in

by t1m 12y ago

Here is a link to some more 'thorough' benchmarks that compare WiredTiger to many other database back-ends. http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/

hyc_symas 12y ago
I'd say the MongoDB folks made a pretty safe choice here, since WiredTiger offers both LSM and Btree engines. I've recently completed some more benchmarks on all of the above-tested engines, and will be publishing the report soon (as soon as it's done being written...). Testing on-disk, with DBs 5x larger than RAM, at data sizes of 24, 96, 384, 768, 2000, and 4000 bytes per record. The important result to note is that LSM's high write speed advantage only exists for small record sizes, and for short durations. On long duration tests the compaction overhead stops them in their tracks, and at data sizes above 2000 bytes per record, LSM write amplification is too expensive (regardless of test duration). I.e., above 2000 byte records, Btrees are always faster than LSMs. This result is particularly relevant for MongoDB, since a document store tends to have large records containing multiple fields, as opposed to individual values being stored as separate small records. And in cases where that applies (large records), nothing comes anywhere close to LMDB's performance. (A preview of the on-disk test results I mentioned was presented at BuildStuff.LT last week http://symas.com/mdb/20141120-BuildStuff-Lightning.pdf http://symas.com/mdb/20141120-BuildStuff-Lightning.pdf pages 103-on)
- eis 12y ago
  Howard I always found your benchmarks as well as your comments and explanations of the results to be very thorough and easy to understand. So thanks for that. I wonder if you had a look at ForestDB which is also based on a B+-tree like structure instead of LSM and claims to have very low write amplification. They have published benchmarks[1] but there is no LMDB or WiredTiger included. [1] https://github.com/couchbaselabs/forestdb/wiki/Performance-Results https://github.com/couchbaselabs/forestdb/wiki/Performance-R...
  - hyc_symas 12y ago
    I attempted to test it a while ago; the test driver is included along with all the others https://github.com/hyc/leveldb/tree/benches/doc/bench https://github.com/hyc/leveldb/tree/benches/doc/bench It crashed in all of my test invocations though, so I haven't looked at it since.
  - hyc_symas 12y ago
    By the way, for those of you who are happy with just the raw data and coming to your own conclusions, you can get the advance look here: http://symas.com/mdb/ondisk/ http://symas.com/mdb/ondisk/
    - hyc_symas 12y ago
      Page now updated with formatted results and conclusions.
  - hyc_symas 12y ago
    Gee, someone else collected all my forestDB comments together... http://www.reddit.com/r/programming/comments/2iau1n/forestdb_a_fast_keyvalue_storage_engine_based_on/cl1dx3m http://www.reddit.com/r/programming/comments/2iau1n/forestdb... With perfect candor, I believe forestDB's announcement was extremely premature.

by t1m 12y ago

Here is a link to some more 'thorough' benchmarks that compare WiredTiger to many other database back-ends. http://symas.com/mdb/inmem/ http://symas.com/mdb/inmem/

hyc_symas 12y ago

I'd say the MongoDB folks made a pretty safe choice here, since WiredTiger offers both LSM and Btree engines. I've recently completed some more benchmarks on all of the above-tested engines, and will be publishing the report soon (as soon as it's done being written...). Testing on-disk, with DBs 5x larger than RAM, at data sizes of 24, 96, 384, 768, 2000, and 4000 bytes per record. The important result to note is that LSM's high write speed advantage only exists for small record sizes, and for short durations. On long duration tests the compaction overhead stops them in their tracks, and at data sizes above 2000 bytes per record, LSM write amplification is too expensive (regardless of test duration). I.e., above 2000 byte records, Btrees are always faster than LSMs. This result is particularly relevant for MongoDB, since a document store tends to have large records containing multiple fields, as opposed to individual values being stored as separate small records. And in cases where that applies (large records), nothing comes anywhere close to LMDB's performance. (A preview of the on-disk test results I mentioned was presented at BuildStuff.LT last week http://symas.com/mdb/20141120-BuildStuff-Lightning.pdf http://symas.com/mdb/20141120-BuildStuff-Lightning.pdf pages 103-on)

eis 12y ago
Howard I always found your benchmarks as well as your comments and explanations of the results to be very thorough and easy to understand. So thanks for that. I wonder if you had a look at ForestDB which is also based on a B+-tree like structure instead of LSM and claims to have very low write amplification. They have published benchmarks[1] but there is no LMDB or WiredTiger included. [1] https://github.com/couchbaselabs/forestdb/wiki/Performance-Results https://github.com/couchbaselabs/forestdb/wiki/Performance-R...
- hyc_symas 12y ago
  I attempted to test it a while ago; the test driver is included along with all the others https://github.com/hyc/leveldb/tree/benches/doc/bench https://github.com/hyc/leveldb/tree/benches/doc/bench It crashed in all of my test invocations though, so I haven't looked at it since.
- hyc_symas 12y ago
  By the way, for those of you who are happy with just the raw data and coming to your own conclusions, you can get the advance look here: http://symas.com/mdb/ondisk/ http://symas.com/mdb/ondisk/
  - hyc_symas 12y ago
    Page now updated with formatted results and conclusions.
- hyc_symas 12y ago
  Gee, someone else collected all my forestDB comments together... http://www.reddit.com/r/programming/comments/2iau1n/forestdb_a_fast_keyvalue_storage_engine_based_on/cl1dx3m http://www.reddit.com/r/programming/comments/2iau1n/forestdb... With perfect candor, I believe forestDB's announcement was extremely premature.