3 ms·
Eric Zhu has a Python library called datasketch also. The docs are good and I found the code a great help in understanding implementation details of some method
by philipkglass 9y ago
Eric Zhu has a Python library called datasketch also. The docs are good and I found the code a great help in understanding implementation details of some methods. I am not using his code in production but it was a great outline for how to implement certain techniques I wanted to use in a Scala project.
https://ekzhu.github.io/datasketch/index.html https://ekzhu.github.io/datasketch/index.html
https://github.com/ekzhu/datasketch https://github.com/ekzhu/datasketch
I don't know how how CountMinSketch compares with Zhu's HyperLogLog and HyperLogLog++ for cardinality counting, in terms of speed/memory/accuracy. But if you're interested in Bounter you may well be interested in Zhu's code too.
- Radim 9y agoInteresting link, thanks. HyperLogLog is a simpler algo in a way, as it only estimates the set cardinality ("count distinct"), as opposed to frequencies of individual items. Bounter includes HyperLogLog as well (mentioned in the blog post); it comes for free with its optimized CountMinSketch.
- philipkglass 9y agoGood point. I mention it because Zhu's code, the Yahoo datasketch code (mentioned in another comment), and Bounter all seem like they may be of interest if you are trying to solve certain classes of problems or just like probabilistic algorithms.