6 ms·
Quickly find sensitive files in your GitHub repo
- yawnxyz 4y agoAre there any tools that check whether you're pushing keys and secrets BEFORE you've uploaded them to Github and it's already too late? E.g. checks your commits, and gives you a massive warning when you've committed something you're not supposed to? Even better if it prevents your commit from going through unless you force it
- itsmemattchung 4y agoHow about a pre-commit hook? https://github.com/pre-commit/pre-commit-hooks#detect-aws-credentials https://github.com/pre-commit/pre-commit-hooks#detect-aws-cr... Although a pre-commit hook exists for detecting AWS credentials, I can imagine other checks (e.g ssh credentials, passwords, etc)
- deleted 4y ago[deleted]
- bdhcuidbebe 4y agoYou can use commit hooks to script git
- Fudgel 4y agoFor people in the JS ecosystem theres this eslint rule: https://github.com/nickdeis/eslint-plugin-no-secrets https://github.com/nickdeis/eslint-plugin-no-secrets
- arkadiyt 4y agoGithub itself has a product like this. It's better than the pre-commit hook approach since developers can turn that off (or never install it in the first place if you don't roll it out via MDM), and this centralized approach can't be bypassed. But, even if you're on their enterprise plan it's still an additional addon (and expensive): https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/about-secret-scanning#about-secret-scanning-for-advanced-security https://docs.github.com/en/enterprise-cloud@latest/code-secu...
- pabs3 4y agoThere are a ton of them, here are a few: https://martinfowler.com/articles/session-secret.html https://martinfowler.com/articles/session-secret.html https://github.com/trufflesecurity/trufflehog https://github.com/trufflesecurity/trufflehog https://github.com/awslabs/git-secrets https://github.com/awslabs/git-secrets https://github.com/sirwart/ripsecrets https://github.com/sirwart/ripsecrets https://thoughtworks.github.io/talisman/ https://thoughtworks.github.io/talisman/ https://github.com/michenriksen/gitrob https://github.com/michenriksen/gitrob
- banana_giraffe 4y agoOne thing to be aware of: Tools like this (and indeed, this one specifically) tend to take what I can only describe as "what AWS keys look like _now_" mentality to how they construct regexps. I learned this the hard way due to a script bouncing around an internal gitlab repo for ages with hardcoded AWS keys, and the second someone accidently published it to a public gist, we learned very quickly people that will abuse public AWS keys actually use AWS's recommended regexp ( https://aws.amazon.com/blogs/security/a-safer-way-to-distribute-aws-credentials-to-ec2/ https://aws.amazon.com/blogs/security/a-safer-way-to-distrib... ) to find AWS keys, even old ones, though it does result in more noise, which is no doubt why tools like this try to be more specific.
- trinovantes 4y agoOne technique API providers could adopt is prefix all keys with a description so there's no ambiguity between keys and random base64 strings and it would make automated checks a lot simpler: e.g. amazonsecret-... githubsecret-... donotcommit-... etc.
- FrenchyJiby 4y agoYou're definitely on to something. I know (personal) Github API tokens use a prefix of ghp_ and the Github-app JWT-issued tokens have another prefix (maybe gha_? Can't remember) As mentioned recently here on HN, a generalized version of this technique is what RFC8959[1] proposes via "secret-token" prefix. Do add these to all your pre-commit hooks and other password detectors/log analysers! [1]: https://www.rfc-editor.org/rfc/rfc8959.html https://www.rfc-editor.org/rfc/rfc8959.html
- arkadiyt 4y agoThis is precisely why github switched their token format and what they recommend other service providers do: https://github.blog/2021-04-05-behind-githubs-new-authentication-token-formats/ https://github.blog/2021-04-05-behind-githubs-new-authentica...
- jdorfman 4y agoI agree, some do but more should dare I say need to follow suit.
- celdon25 4y agoNo relevant results for "master.key" (first and only thing I tried searching for), so I assume they have figured out some awesome way to detect these files based on their contents. Or the heuristics are closed-source. https://github.com/search?q=org%3Asourcegraph+master.key&type=code https://github.com/search?q=org%3Asourcegraph+master.key&typ... https://medium.com/@thorntonbrenden/rails-and-the-legendary-master-key-15c8be7799f1 https://medium.com/@thorntonbrenden/rails-and-the-legendary-...
- tasuki 4y agoI've never accidentally committed a secret into any repository. When I create a file that holds secrets, first thing it either gets encrypted or goes into gitignore. What exactly is the path that leads to committing secrets, apart from utter incompetence or disregard? Do people not review what they're committing at all?
- deleted 4y ago[deleted]
- blep_ 4y agoEvery time I've done this, I've done so knowingly and intentionally, into a repository that is not public and will never be public. That last part is what makes it okay. I can only assume the kind of person who puts their dotfiles in a public github repository - or, more generally, puts things on github by default - does not have as strong a public/private distinction as I do.