5 ms·
GitLeaks – Search engine for exposed secrets on GitHub
- bjarneh 10y agoDidn't take long from the proggit/HN 'removed password' post to gitleaks: $ whois gitleaks.com | grep Creation Creation Date: 06-feb-2017
- julianwachholz 10y agoGitHub searches that expose secrets have been posted numerous times already in the past.
- koolba 10y agoAre there any legal ramifications for operating something like this? I know it's publicly available info but since the original creator of the information didn't directly give it to you, do you still have the usual immunity given to service providers? Also, just because something is on $PUBLIC_URL doesn't mean the copyright would allow you redistribute it. I'm sure a lot of these projects have either a private license, or more likely, no license at all.
- flipp3r 10y agoI guess that's one way to get attention to your business. Instead of informing the owners of repositories by creating an issue, you create a search engine to expose them, and then ask to be paid for usage of this index? The only reason someone would want those secrets is to abuse them. This is basically the only use case for the data. Why do this? This is coming from "fallible.co" whose homepage says "Prevented 40 million+ users personal data leaks". So you are in the business of making sure people's information does not get leaked, and at the same time expose people's secrets?
- deleted 10y ago[deleted]
- kiallmacinnes 10y agoThere are non-abusive uses of this kind of data, e.g. security researchers, or IT departments outsourcing credential leak scanning, etc.. Also, notifying via a GitHub issue is, in my opinion, a terrible idea. GitHub has no concept of a security issue viewable only to the repo maintainers, so filing a public issue might make things worse (by calling public attention to it). A paid search engine without any notification is probably worse, but maybe they are emailing the repo's committers? They may even be embargoing the search results for a period of time.
- flipp3r 10y agoTheir FAQ indicates they update the index every two months, but no information on what they do with the data in the meantime. Ideally this scanner would be a feature of a Github or a Bitbucket or a Gitlab, etc, itself. They could've decided to contact them to add this as a feature, or decided to contact repository owners, but instead they decided to sell the data publicly. Real shame.
- kiallmacinnes 10y agoI do remember reading something about GitHub preventing some keys from being pushed (AWS secrets etc?) But it's a vague memory from a long time ago!
- johncolanduoni 10y agoIf posting individual issues in each project, likely to be seen first by contributors is a bad idea, how is creating a paid search engine likely to be used by people who specifically want to find secrets and not likely to be used/seen by contributors a good thing?
- kiallmacinnes 10y agoI didn't say it was a good thing (or a bad thing), only that there are legitimate use cases and that the suggested notification method would be, in my opinion, terrible security practice...
- nitza 10y agoWhy not use your knowledge of these exposed secrets for good? You know which repo they're coming from, it'd be super simple to let the owner know rather than potentially costing them time and money. It also seems as though the only use of this site is to capitalise on other people's mistakes? It looks like you're just handing over leaked data to people who will definitely abuse it, which seems to go against your core business of preventing data leaks?
- lrusnac 10y agoI agree with what you wrote. My two cents are: seems that getting attention goes against being nice to others, such a shame living in such a society.
- tokenizerrr 10y agoYet if people don't know the risk exists, they'll continue being ignorant and fucking up. Awareness is a good thing.
- nitza 10y agoThis isn't awareness though. This is like telling a specific set of people about all the houses near by that have their front door key under the mat. You only become aware of the issue when it is too late.
- koolba 10y agoWould it be considered spamming to pull the email address of the commits and send them an automated email?
- nitza 10y agoWell, you don't need to send an email necessarily, a GitHub issue with a guide on how to include sensitive data in a public repo would probably suffice.
- johncolanduoni 10y ago
- sarreph 10y agoWhat a shame that you had to expose everyone's mistakes like this in such a blanket fashion. You could've taken the moral high ground and created a reverse-search such as HaveIBeenPwned[0], whereby you check repos you own. I hope this gets taken down because the potential for abuse is ripe. [0] - https://haveibeenpwned.com https://haveibeenpwned.com
- lrusnac 10y agothat's such a great idea, although I think people that would sign up for this service would know not commit credentials
- the_duke 10y agoWell, the service could auto-create an issues for each repository that contains secrets.
- lrusnac 10y agowell, that's kinda the same, facilitating finding the secrets since it's easy to search for issues, feel like a private notification is way better
- Xylakant 10y agoWe all make mistakes despite better knowledge. I'd probably sign up.
- lrusnac 10y agosure, but it's more likely for somebody that doesn't know about this service to publish a secret than someone that is aware of it
- ta_dhee 10y agoI think this is great work, the secrets are already scraped and compromised anyway. Good way to make it more clear. It reminds me of https://twitter.com/dumpmon https://twitter.com/dumpmon on Twitter.
- macca321 10y agoEthics of a business model that notifies owners of the breach, but they have to pay $10 for specific details or wait a week?
- ulucs 10y agoI think a more ethical way to go forward with this would be the haveibeenpwned way, where you can search your email and see where your stuff has been leaked instead of a searchable index of leaks.
- ivanhoe 10y agoProblem is that often project owners will not know/care about joining such services. Unlike the passwords, project's security is a matter of interest for a much wider public (all current and the potential future users), but if you let anyone subscribe to any leak than you are back at square one, cause bad guys can do it too.
- mkagenius 10y agoWoah, too many negative comments here. We wanted to model it like Shodan where we would provide a searchable interface for secrets on the web, starting with GitHub. We are removing the search functionality and account upgrades right now until we can come up with a better solution to inform people about secret leaks. For now, you can simply use the existing Check my GitHub button to scan your public repos.
- Fifer82 10y agoI don't have a problem with it. The only reason you are getting negative comments is that there will be a few HN members biting their nails.
- sschueller 10y agoThe data is public, there is absolutely nothing wrong with this and you should put it back online.
- innerspirit 10y agoMost of the data on there is not meant to be public. It's just a tool to abuse people's ignorance, disguised as a "research tool".
- Fifer82 10y agoYet public it is.
- austincheney 10y agoI absolutely agree. Most of the negative comments on here are nothing more than apologists for incompetence. Like with all security research the solution is to expose the defect to the world. If people didn't want their passwords and keys exposed to the world then they shouldn't put them on public github for all the world to see. It would be more helpful though if such a search engine could auto create an issue on github when exposed secrets come up in a search result.
- 10y ago
- hahamrfunnyguy 10y agoSorry OP, but this is pretty terrible idea. I know the secrets are already out there, but the least that could be done is let the user know about it. I am glad to see the search was taken down. There's nothing wrong with the search, but a better use of it would be to educate and inform. I'd be curious to see which kinds of developers are the most likely to leak sensitive data.
- romanovcode 10y agoHuh, that was quick!
- popey456963 10y agoAn interesting aside whilst the search is down, your styling seems a tiny bit messed up for me on the homepage [0] I'm running Chrome, Win 7 on a mildly large display, nothing particularly out of the ordinary. [0] https://puu.sh/u7htR/3fe6b4725d.png https://puu.sh/u7htR/3fe6b4725d.png
- ascendantlogic 10y agoThis is one of those times where you ask yourself "I know I can do this, but should I?". Most of us know we can search GitHub for stuff like AWS_ACCESS_KEY_ID but putting the work into creating a productized interface for it seems a bit beyond the pale to me.
- sublimino 10y agoOpen source alternatives for Git repos (ideally run in the pipeline): https://github.com/dxa4481/truffleHog https://github.com/dxa4481/truffleHog - "Searches through git repositories for high entropy strings, digging deep into commit history" https://github.com/ezekg/git-hound https://github.com/ezekg/git-hound - "Hound is a Git plugin that helps prevent sensitive data from being committed into a repository by sniffing potential commits against PCRE regular expressions" https://github.com/michenriksen/gitrob https://github.com/michenriksen/gitrob - "The tool will iterate over all public organization and member repositories and match filenames against a range of patterns for files that typically contain sensitive or dangerous information" https://github.com/awslabs/git-secrets https://github.com/awslabs/git-secrets - "Prevents you from committing passwords and other sensitive information to a git repository"
- wybiral 10y agoWeird. Just yesterday this made it to the HN front page (related): https://news.ycombinator.com/item?id=13650818 https://news.ycombinator.com/item?id=13650818
- nydrewreynolds 10y agoI accidentally pushed keys to github last year and got an email from HelpfulOwl letting me know they found them and to remove them. That's an example of using this tech for good.
- mostafaberg 10y agoI don't think it's illegal or wrong to have that search, those mistakes are made by developers who aren't paying attention to security, and from experience those leaks will never be resolved UNTIL they get widely exposed, until then, lots of those people will just shrug it off.. you're actually doing a favour to the users who depend on those developers, you never know when the next leak will be and it might be stopped by forcing the developers to fix it. it's not about you, your service, nor the companies, developers who are leaking secrets like that, it's about the end users and people who are affected, my two cents, put the search back, expose it, it's already exposed and probably black hats already have the secrets and don't want to notify the devs of their mistakes. anyone else who disagrees with you doesn't really understand how big this is, your service is amazing and I totally appreciate the work you did, if someone thinks you're "getting attention" or "evil" they really are not looking at the big picture, the "evil" ones are the people who already have that leaked data and keep it for their personal use.
- LyndsySimon 10y ago'tis better to be pwned and found out than to never realize that you've been pwned at all. - Shakespeare or something
- deleted 10y ago[deleted]
- ivanhoe 10y agoBad guys already have scrapers like this, for years, so this is really not putting anyone at any additional risk. They're already in danger, just not aware of it. Even script-kiddies have cheap tools available to scan repos easily. As I see it the only new angle here is that this service lets ordinary users and other interested parties search for the f* ups and (if they care enough) let the project maintainers know about it.
- hackingNerd 10y agoI support disclosure on time :)