8 ms·
Forgive me if this is a dumb question, but is this library a way of merging multiple HyperLogLogs together and still have a reasonable cardinality estimate? Wh
by djhworld 9y ago
Forgive me if this is a dumb question, but is this library a way of merging multiple HyperLogLogs together and still have a reasonable cardinality estimate?
What sort of loss do you see the more HyperLogLogs you merge? As in, if you had Set1,2,3,4,5.....n
- seiflotfy 9y agoits meant for merging and intersecting and it should be max 5% error
- gopalv 9y ago> a way of merging multiple HyperLogLogs together and still have a reasonable cardinality estimate? HLLs can be merged together with the same sort of result as if you ran the whole set over the same HLL. The merge can't do better than that, but it can do that even if you mess around a bit with the HLL sizes (like, HIVE-18079 is fixing p=14 bits to be p=10 accuracy by default, while retaining old HLLs as-is).