3 ms·
For most of these systems, the answer will depend on how much hardware (disks and RAM) you're throwing at the problem. The RAM-to-disk ratio is a big factor.
by nuggetwheat 16y ago
For most of these systems, the answer will depend on how much hardware (disks and RAM) you're throwing at the problem. The RAM-to-disk ratio is a big factor. If your entire data set fits in memory, the latency will be measured in microseconds. If none of your data fits in memory and each request requires a disk seek, the average latency will be closer 10 milliseconds (assuming there are enough disks to keep up with the requests). Take a look at average latency section in the following benchmark comparing Hypertable with HBase: http://www.hypertable.com/pub/perfeval/test1/ http://www.hypertable.com/pub/perfeval/test1/
Hypertable average latency goes from about 1.5 milliseconds down to about .5 milliseconds. These numbers are average, however, some requests hit disk and were close to 10 milliseconds, but many requests were served out of the cache. The read request pattern also makes a big difference. Read patterns with locality (e.g. Zipfian distribution) will have much a much better cache hit percentage. -Doug
- ghotli 16y agoThank you for the numbers about hypertable. In my exploration I've been leaning towards hypertable but I haven't done any benchmarking of my own yet. My read patters will probably have locality but I have not yet figured out how to model the data in a key / value style to preserve the locality of the data. Can you describe what you mean by Zipfian distribution?
- smallpotatoes 16y agoAn easy way to think about Zipfian distributions is something like "the most popular 30% of your data set is queried 70% of the time". Its basically the temporal locality property that a lot caching systems exploit. For more look at http://en.wikipedia.org/wiki/Zipf%27s_law http://en.wikipedia.org/wiki/Zipf%27s_law and www.icis.ntu.edu.sg/scs-ijit/1204/1204_6.pdf
- ghotli 16y agoThat paper was great. Thanks.