3 ms·

Show HN: InBloom – Portable Bloom Filters in Java, Python and Go

45 points by dvirsky 11y ago

avitzurel 11y ago
Everything.Me have been doing great open source work for a while now. redash (https://github.com/EverythingMe/redash https://github.com/EverythingMe/redash) is another great example. I love seeing open source coming from companies under the company repo and not only from developers, this means a lot looking at a company, it means they actually care about the ecosystem and the community around the languages they use. Kudos to the team there!
- dvirsky 11y ago
  Thanks! We figure, if we write infrastructure stuff that is not part of our core IP, we might as well open it so it won't rot, or at least rot much slower - and you might even get improvements to what you do. Even if it's not a big project like redash, it still helps if you get occasional bug reports and pull requests. Another added bonus - people tend to write higher quality code if they know it's going to be open. Here are a few of our other projects besides redash and Inbloom: Magneto - Android test automation https://github.com/EverythingMe/magneto https://github.com/EverythingMe/magneto Jitt - Just In Time translations for Android apps https://github.com/EverythingMe/jitt https://github.com/EverythingMe/jitt GoFigure - easy configuration file parsing for go https://github.com/EverythingMe/gofigure https://github.com/EverythingMe/gofigure Plaxien - nice "explain" interface for in-app reporting in Android https://github.com/EverythingMe/plaxien https://github.com/EverythingMe/plaxien Geodis - a redis based geo-resolving and geo search library in Python https://github.com/EverythingMe/geodis https://github.com/EverythingMe/geodis Rainbow - AWS Cloudformation on steroids https://github.com/EverythingMe/rainbow https://github.com/EverythingMe/rainbow OpenSpace - a nice dashboard to showcase our open source projects. https://github.com/EverythingMe/openspace https://github.com/EverythingMe/openspace (the full list of projects is demoed on it here - http://everythingme.github.io/ http://everythingme.github.io/)
  - itamarhaber 11y ago
    May you have a long and rot-free life!
jhgg 11y ago
I wrote something very similar a week ago actually, https://github.com/jhgg/pybloof https://github.com/jhgg/pybloof I'm pretty sure I messed it up slightly though. :|
simonw 11y ago
What a great idea for a library. Thanks for sharing.
- dvirsky 11y ago
  Thanks :)
taliesinb 11y ago
Not to take away from this at all, but if you're interested in Bloom filters you might also be interested in Cuckoo filters: https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf, previously on HN here: https://news.ycombinator.com/item?id=8489971 https://news.ycombinator.com/item?id=8489971
JohnDoe365 11y ago
What is the relationship to algorithms like HyperLogLog? They do also estimate set cardinality and seem even more capable?
- dvirsky 11y ago
  HLL can only estimate cardinality, it can't test for the existence of a key. The idea is the same - sacrifice some precision and gain a huge memory saving. But they don't share anything beyond that in terms of the math.

45 points by dvirsky 11y ago

Everything.Me have been doing great open source work for a while now. redash (https://github.com/EverythingMe/redash https://github.com/EverythingMe/redash) is another great example. I love seeing open source coming from companies under the company repo and not only from developers, this means a lot looking at a company, it means they actually care about the ecosystem and the community around the languages they use. Kudos to the team there!

dvirsky 11y ago
Thanks! We figure, if we write infrastructure stuff that is not part of our core IP, we might as well open it so it won't rot, or at least rot much slower - and you might even get improvements to what you do. Even if it's not a big project like redash, it still helps if you get occasional bug reports and pull requests. Another added bonus - people tend to write higher quality code if they know it's going to be open. Here are a few of our other projects besides redash and Inbloom: Magneto - Android test automation https://github.com/EverythingMe/magneto https://github.com/EverythingMe/magneto Jitt - Just In Time translations for Android apps https://github.com/EverythingMe/jitt https://github.com/EverythingMe/jitt GoFigure - easy configuration file parsing for go https://github.com/EverythingMe/gofigure https://github.com/EverythingMe/gofigure Plaxien - nice "explain" interface for in-app reporting in Android https://github.com/EverythingMe/plaxien https://github.com/EverythingMe/plaxien Geodis - a redis based geo-resolving and geo search library in Python https://github.com/EverythingMe/geodis https://github.com/EverythingMe/geodis Rainbow - AWS Cloudformation on steroids https://github.com/EverythingMe/rainbow https://github.com/EverythingMe/rainbow OpenSpace - a nice dashboard to showcase our open source projects. https://github.com/EverythingMe/openspace https://github.com/EverythingMe/openspace (the full list of projects is demoed on it here - http://everythingme.github.io/ http://everythingme.github.io/)
- itamarhaber 11y ago
  May you have a long and rot-free life!

jhgg 11y ago

I wrote something very similar a week ago actually, https://github.com/jhgg/pybloof https://github.com/jhgg/pybloof I'm pretty sure I messed it up slightly though. :|

simonw 11y ago

What a great idea for a library. Thanks for sharing.

dvirsky 11y ago
Thanks :)

taliesinb 11y ago

Not to take away from this at all, but if you're interested in Bloom filters you might also be interested in Cuckoo filters: https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf, previously on HN here: https://news.ycombinator.com/item?id=8489971 https://news.ycombinator.com/item?id=8489971

JohnDoe365 11y ago

What is the relationship to algorithms like HyperLogLog? They do also estimate set cardinality and seem even more capable?

dvirsky 11y ago
HLL can only estimate cardinality, it can't test for the existence of a key. The idea is the same - sacrifice some precision and gain a huge memory saving. But they don't share anything beyond that in terms of the math.