24 ms·
I'm Open Sourcing the Have I Been Pwned Code Base
- mikorym 6y agoDidn't Troy want to sell to someone just a few years ago?
- teh_klev 6y agoAs mentioned in the article in the first paragraph: "and it took a failed M&A process to get here" Second paragraph: "especially in the wake of the M&A process[0] that ended earlier this year right back where I'd started" [0]: https://www.troyhunt.com/project-svalbard-have-i-been-pwned-and-its-ongoing-independence/ https://www.troyhunt.com/project-svalbard-have-i-been-pwned-...
- snapetom 6y agoYes. Just last year. Sounds like this action is a result of that failure. https://threatpost.com/troy-hunt-sell-have-i-been-pwnd/145565/ https://threatpost.com/troy-hunt-sell-have-i-been-pwnd/14556...
- 42droids 6y agoWow this is awesome news. I always wondered how the internals could work. Can’t wait to take a peak...
- FlorianRappl 6y agoYou will be very disappointed ...
- atxbcp 6y agoOne of the tweets in the article says Github is open-source: it's not.
- Techbrunch 6y agoTrue but FYI if you want to, you can reverse engineer the code of the GitHub enterprise version: https://blog.orange.tw/2017/01/bug-bounty-github-enterprise-sql-injection.html https://blog.orange.tw/2017/01/bug-bounty-github-enterprise-...
- rvnx 6y agoOlder versions of GH Enterprise even don't have ruby_concealer You just have an open VM
- johannes1234321 6y agoThe fact that you can (probably under break of license terms) get to source, doesn'tale it open source, as understood by majority of the community (which is somewhere close to OSI's definition)
- dustinmoris 6y agoTL;DR: I tried to sell HIBP for a nice premium to make profit of data breaches, but because the sale fell through And I’m not going to make as much money from HIBP as I’d like, I lost interest in maintaining it and therefore now rebrand it as if it was always a community project, so other people can work on it now.
- lowkeyokay 6y agoSeems fair enough
- rsa25519 6y agoHonestly, maintainer burnout sucks. People volunteer their time, become pillars of modern tech. Then they look around, and everything's just boring. Nobody wants to pay them, and nobody really wants to take on the work for themselves, which is understandable :-/
- nickjj 6y agoThis is why I think directly tying money into open source code* isn't a good idea. It breeds burn out. Instead of creating something with the purest intent of doing it for sheer joy or scratching your own itch, suddenly everything becomes about money and now every second you spend on the project gets evaluated as a way to maximize monetary gains and if things don't work out to your internal expectations you're constantly comparing yourself to others or thinking negatively about how much time you've put into something and how little compensation you're getting from it, which is a horrible feeling. * The code aspect is very important. I'm all for figuring out ways to generate income around open source projects, but if the code is the core of everything and your core is only moving forward because you want money, you're setting yourself up for failure, burn out or worse.
- ChrisSD 6y ago> the purest intent of doing it for sheer joy or scratching your own itch... ...is great for as long as the joy or itch lasts. But the same problem still occurs, at some point a single maintainer is going to lose enthusiasm or have other itches they want to scratch. This becomes especially true when maintenance starts to feel more like a chore. And in open source projects it can get even worse. Github users can be especially demanding and there are often those who express their demands in very "confrontational" ways. Faced with that it can be difficult to remain invested.
- throwaway77384 6y agoWhat is this M&A process he keeps referring to?
- MattGaiser 6y agoHe tried to sell it at one point. https://threatpost.com/have-i-been-pwned-no-longer-for-sale/153401/ https://threatpost.com/have-i-been-pwned-no-longer-for-sale/...
- throwaway77384 6y agoThanks!
- ragebol 6y agoMergers & Acquisitions
- throwaway77384 6y agoThanks!
- rethab 6y agoFor the lazy ones searching for a github link: It is not open source yet: > HIBP isn't in a state to simply flick the visibility of it in GitHub, but it needs to get to that point. Instead, I need to choose the right parts of the project to open up in the right way at the right time. and then further down: > I don't have a timeline for each step along the way yet as HIBP remains something I do in my spare time and I've always got a bunch of other stuff on my plate, but the process has already begun and I'll be sharing more on that as soon as I can.
- detaro 6y agoIt's an interesting question: Could some open group etc replicate it entirely, e.g. something like Let's Encrypt (which is also a free service funded by various companies)? The data sources and import are the key bit where trust is an issue, and weirdly enough a single individual might have an easier time than a well-funded foundation etc.
- anaganisk 6y agoIm not sure if those corporate institutions will allow it to notify breaches quicker, if HIBP is backed by likes of Facebook, google etc. Its better for HIBP to stay away from them.
- aspenmayer 6y agoI asked in comments on post which open source license he picked. Will update if he replies.
- tokai 6y agoHopefully it'll be a proper free one.
- aspenmayer 6y agoHopefully it’ll actually be open source, as in free as in money AND free as in beer. That is, I hope the license Troy Hunt picks for Have I Been Pwned is OSI-approved: https://opensource.org/licenses https://opensource.org/licenses
- aspenmayer 6y agoNeedless to say, I hope it’s also free as in speech. Saved the best for last.
- nelsonic 6y agoIs there a reason (Security, Competitive, etc.) that the code wasn't open source from the start?
- buster 6y agoNot wanting to have toxic user requests and issues open for something you do in your spare time could be one good reason.
- oliwarner 6y agoI'm not sure where you get the impression those somehow don't exist if you don't have a bug tracker. They just go to email, or Twitter. For something like this, keeping things private has allowed Troy to work out the best way to do things with complete autonomy. That's really useful if you have the time and resources to get to market on your own. Now it's stable and best security practices are nailed down, he can open it up to a bit of scrutiny and feature bloom.
- msla 6y agoIt's entirely possible to run a Git server which is read-only (for the rest of the world, anyway) and doesn't allow anyone to bother you.
- jcims 6y agoWhat's the point of doing that though? It's just chumming the water for unsolicited feedback.
- majewsky 6y agoIt allows others to easily mirror your code. The original code owner may disappear for any number of reasons (burnout, early retirement, being run over by a bus, etc.).
- msla 6y ago> What's the point of doing that though? It's just chumming the water for unsolicited feedback. You'd do it to prevent feedback.
- jcun4128 6y agoI'm curious what the code would be/how far of an extent it goes. I mean from my experience using it it's a search box, type in email, get results... so is this going to show scrapers or something where the data comes from? Maybe has efficient hash comparisons or something...
- deanclatworthy 6y agoTroy does talk a little bit about the data here, but the codebase is not entirely useful without the data. I don't doubt it can be improved from Troy's one-man-efforts in that area, but the real value of haveibeenpwnd is in the data - and the code is useless without a huge trove of breach data - which I guess is never going to be open.
- sneak 6y agoBreach data is, by definition, open. You just have to dig a little, and sometimes pay criminals. Troy got it; so can you.
- Craighead 6y agoThat implies the same data still exists, which I don't think is true for many breaches.
- prepend 6y agoI think it’s safe to assume that all breached info is archived and available forever. Both as a curiosity, a research tool, and a crime tool. As a hobbyist, I used to have a folder of breached data (ashley Madison was really fun). And if you add in groups like /r/datahoarders, the data will probably be kept someone. The dataset could be reconstructed if needed.
- anaganisk 6y agoYeah make the data available so that we can have AWS Elastic Pwned, and HIBP to be eclipsed by it
- 1f60c 6y agoThere’s no perfect solution here: publish the raw data, and people who haven’t changed their passwords get cracked, or publish hashed (in some way) passwords, and people will crack the passwords themselves. (That said, I guess most crackers will just get the data straight from the source, but still.)
- makach 6y agoIt's good just for the sake of transparency. Maybe he will find someone who is interested in helping maintain and contribute.
- deleted 6y ago[deleted]
- nsarafa 6y agoI use Spark for Mac to make custom shortcuts. It's a must have for anybody who's aiming for optimal efficiency when in front of their MacBooks ~ https://www.macupdate.com/app/mac/14352/spark https://www.macupdate.com/app/mac/14352/spark
- theshrike79 6y agoI give it 24 hours after release before someone rewrites it in Rust. 48 hours before its on HN.
- notmalc 6y agoThis man knows
- yodsanklai 6y agoVery naive question. Suppose I offer a service (e.g. Have I Been Pwned) and I want my users to trust my service not to store/share their data. I understand that open sourcing the code base is one step, but how can I know that the server running the service actually does run this codebase and not something else?
- marksomnian 6y agoThis is a very difficult problem to solve. As far as I know (and am welcome to be corrected!), the current state of the art is remote attestation - Signal wrote a post about how they're doing it for their contact discovery service[0]. It's a very limited solution though. [0]: https://signal.org/blog/private-contact-discovery/ https://signal.org/blog/private-contact-discovery/
- cuu508 6y agoTL;DR they use Intel's SGX. Intel's 20GB data leak was on HN yesterday, with a hint that more is coming. Will be interesting to watch ;-)
- mLuby 6y agoIDK, trusting a remote system just seems bad. Either give me the data to process locally, or I'll give you some data you couldn't possibly understand and you do some stuff to it and return the results, having never known the meaning of the data. In Signal's case, they could just let you send messages that might never be received because the contact doesn't currently have the app. Also gives the receiver plausible deniability unless they respond. And could help get more people on Signal if they did the "Someone sent you a message, download the app now" trick.
- coddle-hark 6y agoFlipping the question around: How do YOU know that the server running the service is actually running the codebase and not something else? You might have been hacked. Someone at your hosting provider might be exfiltrating data. Intel might have a backdoor in their CPUs. It’s not practical for you to continuously verify all of these things, at the end of the day you have to trust that things do what they say they do until you have reason to believe otherwise. Of course, if you can’t verify what’s running in your server then neither can your users. All you can do is be as transparent as possible about how you process their data and hope that’s enough to earn their trust. As an aside, homomorphic encryption has been touted for a while now as a way for services to process encrypted user data without ever being able to decrypt it. Haven’t seen much use of it in the industry though, last I heard it was enormously impractical.
- bawana 6y agoOpen sourcing code is one thing. Open sourcing the email addresses of vulnerable victims is something else. It’s like publishing the largest vulnerability of all time before it can be patched
- dastx 6y agoAll of which is readily available on tor anyway. It's just now centralised.
- boomboomsubban 6y agoAll the email addresses are pulled from public leaks, often from years old leaks. Nothing like your analogy, if you're still vulnerable you should consider yourself lucky you haven't already been hacked.
- BenjiWiebe 6y agoAm I missing something? I understand from the article that he is open sourcing the codebase, not the data...?
- samirillian 6y ago> I'm sure I speak for [Junade] as well when I say we couldn't be happier that other companies have taken the model we pioneered and applied it to their own services too because at the end of the day, that's in everyone's best interests. Socialism for "other companies," capitalism for us. It's details like this that prove to me how purely ideological it is to claim that we need capitalism to "produce value." _We_ produce value already. Capitalism uses that value for free or a ridiculously low rate, then turns around and charges us for it.
- nickthemagicman 6y agoIsn't the data whats important? Isn't the code just essentially a text input box that takes a string, hashes it, and runs it against hashed passwords in a database?
- andruby 6y agoYes, and Troy mentions this in the comments of the blogpost. He's open-sourcing it so the community can help him manage the project, which I think is a fair request/hope.
- mooreds 6y agoYup. Gathering the data is a large part of the value he provides. I wrote an article about how to implement this, and data gathering tasks are crucial: https://fusionauth.io/learn/expert-advice/security/breached-password-detection https://fusionauth.io/learn/expert-advice/security/breached-... He touches on the non technical difficulties as well with his comment: "We invite parties to form their own views on the legality of the data." So the fact he's gathered it all lets HIBP be an service that other companies can use without worrying about the thorny legal question. But there's also his reputation as a steward of the system, which is valuable beyond the data itself. Anyway, while he didn't actually open source anything yet, I'm glad he's committing to it, as hopefully that will allow this internet resource to continue.
- octorian 6y agoI hate to say it, but almost all of these sites/systems end up being far more annoying than useful. Why? Almost all the time, their alerts simply are not actionable. At best, they'll tell you that your username/email was included in a breach and often identify the breach itself by some nebulous data cache name that means nothing to you. They almost never tell you which site was actually breached, nor do they ever give you any hints as to what password was actually compromised. So really, when I show up in one of these alerts, I'm always asking myself: - Was this a recent breach, or a redundant alert from something I dealt with months ago? - What account do I actually need to update, if any, to be safe from this alert? These questions almost never seem to be answered. As someone who uses a password manager and a different random password for every site, there's no way I'm going to proactively hunt down and change every single entry in its DB when I get the "alert of the week." (FWIW, I once worked somewhere that somehow had access to a far better version of this data than they'll ever let the public get access to. That system actually did generate alerts that were actionable. I only wish I had a way to get useful alerts like that as a private individual.)
- newman8r 6y ago> FWIW, I once worked somewhere that somehow had access to a far better version of this data than they'll ever let the public get access to If you really want to hunt it down, a lot of this data is available via torrent, although probably not too useful for generating alerts.
- TimTheTinker 6y ago1Password and other services plug into HIBP to alert you when an account has been pwned and the password should be changed.
- jacoblambda 6y agoIf you use KeepassXC or another password manager of some kind, they are able to tell you which username/password combinations have been compromised.
- jimhefferon 6y ago