3 ms·

Great analysis but can someone fork this and update the graphs to improve some things? I'd do it but I'm on a really bad internet connection right now. 1. Use

Great analysis but can someone fork this and update the graphs to improve some things? I'd do it but I'm on a really bad internet connection right now. 1. Use SI magnitudes, so that we don't have to count the zeros. (so 100k instead of 100000.) 2. Either use the same chart for LSM vs Btree for each of read and write, or at the very least use the same y axis between the two. 3. It's hard to see the difference on the limited write benchmarks. Can we make it either logarithmic or another graph with these differences highlighted?

ddorian43 10y ago
Many benchmarks on different dbs using either lsm/btree on different environments/data-sets: https://symas.com/products/lightning-memory-mapped-database/project-benchmarks/ https://symas.com/products/lightning-memory-mapped-database/... (including wired tiger)
- petergeoghegan 10y ago
  I agree that the quality of each implementation is very pertinent here. I wonder if the B-Tree implementation in question used the Lehman & Yao B-link technique, in order to obviate the need for "latch coupling"/"crabbing". At the same time, it's interesting to consider if the LSM implementation is similarly optimized to minimize low-level locks/semaphore overhead, if in fact that's actually possible. In general, B-Tree index performance has plenty to do with low-level optimizations, beyond things that are published in academic papers. I'm talking about things like micro-optimizing the number of CPU cache misses on internal pages, the use of interpolation search, and so on.
edejong 10y ago
4. Please use graphs which highlight the distribution of the write/read latencies. This points to possible outliers and reduces the reliability on averages. Possible example graphs are a violin plot[1] or a scatterplot with jitter [2]. [1] https://en.wikipedia.org/wiki/Violin_plot https://en.wikipedia.org/wiki/Violin_plot [2] http://stackoverflow.com/questions/23675735/how-to-add-boxplots-to-scatterplot-with-jitter http://stackoverflow.com/questions/23675735/how-to-add-boxpl...

ddorian43 10y ago

Many benchmarks on different dbs using either lsm/btree on different environments/data-sets: https://symas.com/products/lightning-memory-mapped-database/project-benchmarks/ https://symas.com/products/lightning-memory-mapped-database/... (including wired tiger)

petergeoghegan 10y ago
I agree that the quality of each implementation is very pertinent here. I wonder if the B-Tree implementation in question used the Lehman & Yao B-link technique, in order to obviate the need for "latch coupling"/"crabbing". At the same time, it's interesting to consider if the LSM implementation is similarly optimized to minimize low-level locks/semaphore overhead, if in fact that's actually possible. In general, B-Tree index performance has plenty to do with low-level optimizations, beyond things that are published in academic papers. I'm talking about things like micro-optimizing the number of CPU cache misses on internal pages, the use of interpolation search, and so on.

edejong 10y ago

4. Please use graphs which highlight the distribution of the write/read latencies. This points to possible outliers and reduces the reliability on averages. Possible example graphs are a violin plot[1] or a scatterplot with jitter [2]. [1] https://en.wikipedia.org/wiki/Violin_plot https://en.wikipedia.org/wiki/Violin_plot [2] http://stackoverflow.com/questions/23675735/how-to-add-boxplots-to-scatterplot-with-jitter http://stackoverflow.com/questions/23675735/how-to-add-boxpl...