3 ms·
Hello! I'm sure a specialized server can use even the latest couple clock cycles at its best, but the Redis implementation performs very reasonably. I ran the f
by antirez 13y ago
Hello! I'm sure a specialized server can use even the latest couple clock cycles at its best, but the Redis implementation performs very reasonably. I ran the following two benchmark simultaneously against my Macbook Air 11" that are able to trigger the worst case scenario from the point of view of the HLL data structure:
./redis-benchmark -P 64 -r 100000000 -n 100000000 \
pfadd hll __rand_int__
pfadd hll __rand_int__: 320125.56
And
./redis-benchmark -P 64 -r 100000000 -n 100000000 \
pfcount hll
pfcount hll: 313309.62
Since we are adding random integers in the range from 0 to 100 million we are basically triggering the worst case in which the updates are as frequent as possible in the registers.
At the same time HLLCOUNT is running in the other benchmark forcing the update of the cached cardinality as often as possible.
Yet we get around 640k ops/sec in a single core, that multiplied for the 4 cores of an Air i7 gets you to 2.4m ops/sec.
EDIT 1: Note that while I'm using random data as argument of PFADD, we are not using N HLLs but a single one, so tis benchmark is surely biased as cache misses are rare. But using multiple keys will not be terrible like losing an order of magnitude, just a reasonable percentage of OPS will go away.
EDIT 2: I had a process running in background consuming a lot of CPU, actually I tried again and reached 900k OPS/sec for core, but read EDIT 1 for more info.
- mitchellh 13y agoYes, Redis performs well. I don't think I ever actually compared to hlld to Redis in my comment, except to say you LOSE data structures by using hlld specifically. I was not making a comparison. I was showing another potential solution.
- antirez 13y agoNo problem at all with comparing! And thanks for commenting. I wrote my message since you wrote "If you're looking for much higher performance than what Redis ...". My idea is just that at the level of performance of the HLL Redis implementation to add a new component to an infrastructure for the sake of faster HLL should be considered with care, but specialized solutions may be better in many regards of course.
- VikingCoder 13y ago"If you're looking for much higher performance than what Redis will get for this specific data structure" That's a comparison, specifically claiming much higher performance.
- mitchellh 13y agoWhoops! Removed. Thanks.