Another common flaw in broken benchmarks: use a single client instead of many simultaneous clients. In this configuration you are mostly measuring the RTT plus

Another common flaw in broken benchmarks: use a single client instead of many simultaneous clients. In this configuration you are mostly measuring the RTT plus the client lib implementation latency. Another one is testing on-disk DBs with datasets that fit in RAM: the whole DB data will be cached in OS buffers and you don't get the picture of how the DB will perform with big data. Benchmarking is one of the most complex tasks, but for some reason benchmarking is usually not done by the most expert guys in the field.