3 ms·

All of those rely on Troy Hunt's API. Which may be fine for some people, yet others may uncomfortable introducing an external dependency. I generally recommend

by hannob 8y ago

All of those rely on Troy Hunt's API. Which may be fine for some people, yet others may uncomfortable introducing an external dependency. I generally recommend avoiding external API dependencies if you can. Here's a python implementation using bloom filters which avoids storing the whole list (need to store ~1 gig), yet still gives you very good accuracy: https://gist.github.com/marcan/23e1ec416bf884dcd7f0e635ce5f2724 https://gist.github.com/marcan/23e1ec416bf884dcd7f0e635ce5f2... It's more what I think this should look like.

hiccuphippo 8y ago
Ha, I was just about to ask for a bloom filter version of this. Thanks!
ThePhysicist 8y ago
Similarly I also wrote a locally hostsable Golang-based REST web service that you can use to check plain passwords or hashes against the HIBP database (and other dbs). It’s based on an optimized Bloom filter library and pretty fast. It also provides a CLI tool and libraries for Python and Go: https://github.com/adewes/have-i-been-bloomed https://github.com/adewes/have-i-been-bloomed
bartread 8y ago
Tend to agree on third party dependencies. They're a useful leg up if you're just prototyping, testing an idea, or generally groping around for product/market fit, but I'm not in love with them as a long term solution - certainly not for anything that's core to your product or service. Too many potential pitfalls: - If their service goes down, you may go down too, or you have the added complexity of gracefully handling the situation when they're down - You have no control over their roadmap... which means you might suddenly get a bunch of non-value-adding but totally essential work dropped into your backlog, and perhaps at short notice, because of changes they make - Perhaps they go out of business or, for whatever reason shut down their service: again, congratulations, you've just got a load of work you didn't bank on getting in the way of delivering your own roadmap I think most of us have probably seen multiple examples of the above, if not at first hand, then posted on HN or elsewhere on the web. I'm not an NIH kind of guy but I do tend to prefer libraries, or co-located installs for long-term dependencies. That way at least you can manage migrations and updates according to your own agenda rather than somebody else's.
matheweis 8y ago
There’s no reason you can’t do both... simply build your internal code to check against multiple sources. You can asynchronously hit your bloom filter, HIBP, DeHashed, etc. and cut short whenever there is a hit. In this way you get the best of all worlds; speed, highest degree of accuracy, and reduced dependency on a single external API.

by hannob 8y ago

hiccuphippo 8y ago

Ha, I was just about to ask for a bloom filter version of this. Thanks!

ThePhysicist 8y ago

Similarly I also wrote a locally hostsable Golang-based REST web service that you can use to check plain passwords or hashes against the HIBP database (and other dbs). It’s based on an optimized Bloom filter library and pretty fast. It also provides a CLI tool and libraries for Python and Go: https://github.com/adewes/have-i-been-bloomed https://github.com/adewes/have-i-been-bloomed

bartread 8y ago

Tend to agree on third party dependencies. They're a useful leg up if you're just prototyping, testing an idea, or generally groping around for product/market fit, but I'm not in love with them as a long term solution - certainly not for anything that's core to your product or service. Too many potential pitfalls: - If their service goes down, you may go down too, or you have the added complexity of gracefully handling the situation when they're down - You have no control over their roadmap... which means you might suddenly get a bunch of non-value-adding but totally essential work dropped into your backlog, and perhaps at short notice, because of changes they make - Perhaps they go out of business or, for whatever reason shut down their service: again, congratulations, you've just got a load of work you didn't bank on getting in the way of delivering your own roadmap I think most of us have probably seen multiple examples of the above, if not at first hand, then posted on HN or elsewhere on the web. I'm not an NIH kind of guy but I do tend to prefer libraries, or co-located installs for long-term dependencies. That way at least you can manage migrations and updates according to your own agenda rather than somebody else's.

matheweis 8y ago

There’s no reason you can’t do both... simply build your internal code to check against multiple sources. You can asynchronously hit your bloom filter, HIBP, DeHashed, etc. and cut short whenever there is a hit. In this way you get the best of all worlds; speed, highest degree of accuracy, and reduced dependency on a single external API.