Hyperloglog is part of a class of algorithms known as “sketches”, which use precomputed probabilistic data structures to perform fast but approximate computatio

Hyperloglog is part of a class of algorithms known as “sketches”, which use precomputed probabilistic data structures to perform fast but approximate computations on very large data sets. Supported computations include count distinct, frequency, sampling, and quantiles and histograms. There’s a project called Apache Datasketches (developed at Yahoo) that implements production versions of these algorithms. They are useful in the implementations of search engines, discussion forum software, etc. that are designed for scale. https://datasketches.apache.org/docs/Background/TheChallenge.html https://datasketches.apache.org/docs/Background/TheChallenge... Another sketch is the well-known Bloom filter, which can quickly test if an element is part of a set without ever returning a false negative (useful for quickly checking a large database for whether a particular username is still available).