6 ms·

Tell HN: Github has dozens of public s3 passwords

57 points by kabuks 16y ago

I almost pushed my s3 credentials to a public github repo for the third time in one day. So, I got curious. Are there people out there who forget cover their tracks? A quick search shows quite a few 'open' buckets out there. What's the best way to warn these folks? What other credentials are lurking out there? Here's the search: http://github.com/search?langOverride=&language=&q=S3+Base.establish_connection&repo=&start_value=1&type=Code&x=0&y=0 And the first open bucket I found: http://github.com/prakashraman/jammmin/blob/a668672c69fafdb8317fec4fb19b7abb0b318e1a/app/scripts/s3_connect.rb

kabuks 16y ago
A more precise search http://github.com/search?langOverride=&language=yaml&q=access_key_id+!ENV+!Panda&repo=&start_value=2&type=Code&x=16&y=20 http://github.com/search?langOverride=&language=yaml&#38...
relix 16y ago
It's something that almost happened to me too. A git repository for a project that at first, in the prototype-stage, had credentials hard-coded, but inevitably moved on to be config-file-driven. I concocted the following command to change all mentions of a specific word to another word, in a git repository: git filter-branch --tree-filter "find . -type f -exec sed -i -e 's/originalpassword/newpassword/g' {} \;" Use this to check if there are any mentions of some word, e.g. your password, in the repository: git filter-branch --tree-filter "grep -r originalpassword * || true"
ritonlajoie 16y ago
" What's the best way to warn these folks? What other credentials are lurking out there?" Sure, posting that on HN is a good way...
- pierrefar 16y ago
  How about leaving them some README-NOW.txt file in their S3 bucket?
  - tlrobinson 16y ago
    I rarely list the contents of my S3 buckets so I'd probably never notice this. Send them a message on GitHub, perhaps?
jeebusroxors 16y ago
How about something with github API? I'm pretty sure you can search/email?
milkshakes 16y ago
why don't you just email github? i'll bet they have a way to contact their users
- mrduncan 16y ago
  http://support.github.com/discussion/new http://support.github.com/discussion/new Don't forget to make it a private discussion.
igorgue 16y ago
Yeah, this is a typical mistake, I've pushed Django's secret keys and even twitter credentials. PS: I do it all the time on my private repos but I try to not do it on the public ones.
- ulf 16y ago
  In case of django there is a nice workaround in using a non-git localsettings.py file. Just put "from localsettings import *" in your settings-file, and keep all deployment-specific settings in the localsettings-file
megafotze 16y ago
yup, there are a lot of credentials out there. you have to watch out what you commit as it will be public viewable in history views.
jah 16y ago
In case anyone else made this mistake, Github's help section has an article describing the steps needed to remove sensitive data from a git repo: http://help.github.com/removing-sensitive-data/ http://help.github.com/removing-sensitive-data/
- DavidSJ 16y ago
  In this case rotating your S3 creds is probably the optimal solution.
adrianscott 16y ago
good reminder to us all, thanks Kabuks
ck2 16y ago
Protip: keep settings in a separate file Mark the file as excluded from svn/git. Make a settings.sample file for the project.
baxter 16y ago
Rather than store passwords or sensitive config data in a file I'll usually try to store them in an environment variable. Heroku has a little guide on this: http://docs.heroku.com/config-vars http://docs.heroku.com/config-vars
- trevorturk 16y ago
  I use a similar strategy: http://trevorturk.com/2009/06/25/config-vars-and-heroku/ http://trevorturk.com/2009/06/25/config-vars-and-heroku/ ...this kind of thing doesn't have to be Heroku-specific, though.
  - ujeezy 16y ago
    Yup, symlinking config files after deploy is the approach I use. Thanks for the Cap task to set things up; at the rate I setup new projects, manually copying those files was getting old :)
jbeda 16y ago
There is a business opportunity here. Managing credentials and secrets is (passwords, private certs, etc) is hard. Making that process turnkey, secure and easy would be useful.
- thenduks 16y ago
  I'm not sure it's that hard. You can use any number of very low-barrier strategies to solve this. One that was mentioned a few times in this thread is to use environment variables. Personally I just put a {whatever}.yml or similar in my project tree somewhere and throw it in .gitignore. I can't imagine a process that starts with "go to westorecredentials.com and sign up..." being any easier. Willing to be surprised :)
  - fragmede 16y ago
    People are advocating not saving settings to source control. F-that, I want to be able to clone my repo and run the project without scp-ing an additional file around. A file, that by all rights should be part of the project. Here's my solution - github 'extends' git somehow, so files can be marked private. Then, unless you have commit rights to that repo, you can't see the file's contents. So a public http checkouts lacks the secret password. And because it's git, someone you give commit rights to is someone you trust. People you don't trust can just make their own repo.
    - thenduks 16y ago
      That sounds like too much software to me. What's with the aversion to scp? You only have to do it once per box you set up... There's something to be said for having GitHub do everything 1000% for you, but there's also something to the unix philosophy that would suggest git is good at the version control thing, and scp is good for the configuration-file-moving thing.
      - fragmede 16y ago
        so... which version of the config file are you using?
pskomoroch 16y ago
There is a "message" button on each github user page. I just sent this to a few people (including someone who had forked one of my projects and added their S3 keys to a config file): Noticed you have your Amazon S3 keys out in the open on github. You might want to remove those config files from your repository as described in the thread here: http://news.ycombinator.com/item?id=1574211 http://news.ycombinator.com/item?id=1574211 -Pete
- gojomo 16y ago
  Remove and change at Amazon, as the values could still be floating around in fetches, caches, archives, indexes, etc. for a while.
  - kabuks 16y ago
    And still cached in their search cache
tlrobinson 16y ago
Might be worth setting up a git hook to scan your commits for sensitive data. Of course you'll need to include that sensitive data in the script, though the first few characters of AWS credentials should be unique enough. I thought about setting up something similar for networking. If a packet contains my password in cleartext then pop up a warning allowing/denying (denying would have to force the connect to close, I guess). Might be too much overhead though.
roder 16y ago
What I have done to prevent that is to use the OS's environment variables: http://github.com/roder/riakaws/blob/master/clouds.rb#L10 http://github.com/roder/riakaws/blob/master/clouds.rb#L10
prakashraman 16y ago
My God, I can't believe I left it there. While I was developing a while ago I had put it there and completely forgot to remove it. What a costly mistake this could've been or even is. Thank you, kabuks, so much for noticing this. I have changed my S3 key pair and am getting to cleaning up my git commits. My God ! But thanks so much.

57 points by kabuks 16y ago

kabuks 16y ago

A more precise search http://github.com/search?langOverride=&language=yaml&q=access_key_id+!ENV+!Panda&repo=&start_value=2&type=Code&x=16&y=20 http://github.com/search?langOverride=&language=yaml&#38...

relix 16y ago

It's something that almost happened to me too. A git repository for a project that at first, in the prototype-stage, had credentials hard-coded, but inevitably moved on to be config-file-driven. I concocted the following command to change all mentions of a specific word to another word, in a git repository: git filter-branch --tree-filter "find . -type f -exec sed -i -e 's/originalpassword/newpassword/g' {} \;" Use this to check if there are any mentions of some word, e.g. your password, in the repository: git filter-branch --tree-filter "grep -r originalpassword * || true"

ritonlajoie 16y ago

" What's the best way to warn these folks? What other credentials are lurking out there?" Sure, posting that on HN is a good way...

pierrefar 16y ago
How about leaving them some README-NOW.txt file in their S3 bucket?
- tlrobinson 16y ago
  I rarely list the contents of my S3 buckets so I'd probably never notice this. Send them a message on GitHub, perhaps?

jeebusroxors 16y ago

How about something with github API? I'm pretty sure you can search/email?

milkshakes 16y ago

why don't you just email github? i'll bet they have a way to contact their users

mrduncan 16y ago
http://support.github.com/discussion/new http://support.github.com/discussion/new Don't forget to make it a private discussion.

igorgue 16y ago

Yeah, this is a typical mistake, I've pushed Django's secret keys and even twitter credentials. PS: I do it all the time on my private repos but I try to not do it on the public ones.

ulf 16y ago
In case of django there is a nice workaround in using a non-git localsettings.py file. Just put "from localsettings import *" in your settings-file, and keep all deployment-specific settings in the localsettings-file

megafotze 16y ago

yup, there are a lot of credentials out there. you have to watch out what you commit as it will be public viewable in history views.

jah 16y ago

In case anyone else made this mistake, Github's help section has an article describing the steps needed to remove sensitive data from a git repo: http://help.github.com/removing-sensitive-data/ http://help.github.com/removing-sensitive-data/

DavidSJ 16y ago
In this case rotating your S3 creds is probably the optimal solution.

adrianscott 16y ago

good reminder to us all, thanks Kabuks

ck2 16y ago

Protip: keep settings in a separate file Mark the file as excluded from svn/git. Make a settings.sample file for the project.

baxter 16y ago

Rather than store passwords or sensitive config data in a file I'll usually try to store them in an environment variable. Heroku has a little guide on this: http://docs.heroku.com/config-vars http://docs.heroku.com/config-vars

trevorturk 16y ago
I use a similar strategy: http://trevorturk.com/2009/06/25/config-vars-and-heroku/ http://trevorturk.com/2009/06/25/config-vars-and-heroku/ ...this kind of thing doesn't have to be Heroku-specific, though.
- ujeezy 16y ago
  Yup, symlinking config files after deploy is the approach I use. Thanks for the Cap task to set things up; at the rate I setup new projects, manually copying those files was getting old :)

jbeda 16y ago

There is a business opportunity here. Managing credentials and secrets is (passwords, private certs, etc) is hard. Making that process turnkey, secure and easy would be useful.

thenduks 16y ago
I'm not sure it's that hard. You can use any number of very low-barrier strategies to solve this. One that was mentioned a few times in this thread is to use environment variables. Personally I just put a {whatever}.yml or similar in my project tree somewhere and throw it in .gitignore. I can't imagine a process that starts with "go to westorecredentials.com and sign up..." being any easier. Willing to be surprised :)
- fragmede 16y ago
  People are advocating not saving settings to source control. F-that, I want to be able to clone my repo and run the project without scp-ing an additional file around. A file, that by all rights should be part of the project. Here's my solution - github 'extends' git somehow, so files can be marked private. Then, unless you have commit rights to that repo, you can't see the file's contents. So a public http checkouts lacks the secret password. And because it's git, someone you give commit rights to is someone you trust. People you don't trust can just make their own repo.
  - thenduks 16y ago
    That sounds like too much software to me. What's with the aversion to scp? You only have to do it once per box you set up... There's something to be said for having GitHub do everything 1000% for you, but there's also something to the unix philosophy that would suggest git is good at the version control thing, and scp is good for the configuration-file-moving thing.
    - fragmede 16y ago
      so... which version of the config file are you using?

pskomoroch 16y ago

There is a "message" button on each github user page. I just sent this to a few people (including someone who had forked one of my projects and added their S3 keys to a config file): Noticed you have your Amazon S3 keys out in the open on github. You might want to remove those config files from your repository as described in the thread here: http://news.ycombinator.com/item?id=1574211 http://news.ycombinator.com/item?id=1574211 -Pete

gojomo 16y ago
Remove and change at Amazon, as the values could still be floating around in fetches, caches, archives, indexes, etc. for a while.
- kabuks 16y ago
  And still cached in their search cache

tlrobinson 16y ago

Might be worth setting up a git hook to scan your commits for sensitive data. Of course you'll need to include that sensitive data in the script, though the first few characters of AWS credentials should be unique enough. I thought about setting up something similar for networking. If a packet contains my password in cleartext then pop up a warning allowing/denying (denying would have to force the connect to close, I guess). Might be too much overhead though.

roder 16y ago

What I have done to prevent that is to use the OS's environment variables: http://github.com/roder/riakaws/blob/master/clouds.rb#L10 http://github.com/roder/riakaws/blob/master/clouds.rb#L10

prakashraman 16y ago

My God, I can't believe I left it there. While I was developing a while ago I had put it there and completely forgot to remove it. What a costly mistake this could've been or even is. Thank you, kabuks, so much for noticing this. I have changed my S3 key pair and am getting to cleaning up my git commits. My God ! But thanks so much.