6 ms·

Blackbox: Safely store secrets in Git

150 points by akerl_ 12y ago

robinhoodexe 12y ago
Interesting. For storing passwords I use Pass[1] which encrypts everything with GPG and then store it using Git in a private bitbucket repo. Works really well for me. I love the simplicity and ease of use. [1]http://www.zx2c4.com/projects/password-store/ http://www.zx2c4.com/projects/password-store/
- akerl_ 12y ago
  One of the things that looked cool about blackbox to me was the support for encrypting the data with multiple keys and managing that. I know some folks who use Pass and recommend it highly, but it seems focused on the personal use case like what you're doing. I do still need to set up Pass myself, and was thinking about making an Alfred workflow to let me quickly access credentials. Does such a thing already exist? (searching for "pass" is not exactly effective :))
  - dnerdy 12y ago
    You can specify multiple gpg ids when using "pass init": pass init [-p sub-folder] user1@example.com user2@example.com This enables you to control access per subfolder. You can keep your personal passwords in one subfolder and then have various shared subfolders. "pass mv" and "pass cp" will reencrypt with the gpg ids for each subfolder as necessary. You can also "pass init" an existing subfolder to add or remove gpg ids for existing files. See the man page for details [1]. [1] http://git.zx2c4.com/password-store/about/ http://git.zx2c4.com/password-store/about/
scosman 12y ago
why not just use openssl or another command line tool? To encode: openssl aes-256-cbc -a -salt -in secret.zip -out secret.zip.enc To decode: openssl aes-256-cbc -d -a -in secret.zip.enc -out secret2.zip
- scosman 12y ago
  and nevermind. The doc explain why (large teams, key sharing, etc)
- elasticdog 12y ago
  That's essentially what I've automated with transcrypt (which uses Git's clean/smudge filters) to do that for you transparently: https://github.com/elasticdog/transcrypt https://github.com/elasticdog/transcrypt
- YesThatTom2 12y ago
  Why not? Why would you want to? Computers exist to make things easier, not harder.
- e12e 12y ago
  Maybe when openssl supports gcm or some other authenticated symmetric cipher mode on the command line?
deadponies 12y ago
Storing anything of value in the cloud, regardless of encryption, is the very essence of stupidity. How many times does this fact need to be reiterated in the news before people get it.
- orblivion 12y ago
  When was it ever in the news that something encrypted was compromised?
  - therealmarv 12y ago
    http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryption-codes-security http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryp...
    - orblivion 12y ago
      I remember this story, though I forgot exactly how deep it went. I thought it was just compromising certificate authorities, or sneaking in physically to catch part of the stream that's not encrypted like they did with Google. That's irrelevant here because the only computers that would hold decrypted info in the case of Blackbox are those computers which would anyway hold it decrypted without Blackbox. So there's still the part about compromising the encryption software. Well, GPG is not proprietary so that's a lot harder. As far as somehow brute forcing the cyphertext, I guess that may be possible, though they don't give clear details about that here. That being the case, they still don't say anything about GPG. Perhaps the biggest argument of all is that Edward Snowden himself uses it, knowing everything he knows.
- Sir_Cmpwn 12y ago
  You could write properly encrypted data on a billboard in Times Square and it would still be secure.
- aftbit 12y ago
  A few years ago, my company was setting up a second office a few hours drive away. They wanted a replicated file share between the two offices. For the initial sync, they drove a NAS from the main office to the satellite. Then they had (openvpn-encrypted) rsyncs on a nightly basis. But the ITAR directory on the NAS was never rsynced. Instead, they drove the files out there every week using a flash drive. Seemed like a crazy policy to me, but after Heartbleed, who knows?
deleted 12y ago
[deleted]
yaccz 12y ago
1. store secretes in separate repo from code itself. Problem solved. 2. You still need to manually decrupt and encrypt files when you want to work with them. 3. It's written in shell. I'll never trust shell programs to be stable. They are too fragile to external changes and will break in runtime. I'm never ever work with secrets with shell wrapper.
vanilla 12y ago
git-crypt[1] also does per file encryption [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/
- agwa 12y ago
  And the upcoming version (which is overdue--I've been busy) will support encrypting with GPG with multiple recipients, like BlackBox.
miles_matthias 12y ago
we use https://github.com/ahoward/sekrets https://github.com/ahoward/sekrets at dojo4. love it.
jordigh 12y ago
Why does the HN title say "in git"? This tool seems to be specifically built to handle hg as well. A pleasant surprise for hg users like me. :-)
- akerl_ 12y ago
  My apologies; I took the title from the name/tagline of the repo. It won't let me edit the title now, but if a mod wanted to do me the favor and make it say "git/mercurial"...
  - jordigh 12y ago
    Oh, I didn't notice the tagline. HN favours the original article's title in most cases, so I suppose a tagline is about as close to a title as we have in this case.
    - YesThatTom2 12y ago
      I've corrected the tag line. Thanks for pointing that out! Ironically Blackbox was originally for a Puppet repo that used Mercurial and only supported Git as a skunkswork project. Now it is devoid of any Puppet dependencies and is mostly used for Git, but the Mercurial support is solid. In fact, you could easily extend it to support other systems. CVS support, anyone?
rphillips 12y ago
Interesting... Very similar to my golang project. https://github.com/rphillips/gosec https://github.com/rphillips/gosec
elasticdog 12y ago
There are a lot of alternatives in this space (I wrote transcrypt [1] for just this purpose), but I think the killer features here are that you don't need shared credentials since you can have multiple keys, and that it's generic enough to work with multiple VCS's. With transcrypt, I do the encryption/decryption transparently once a repository has been configured by utilizing Git's clean/smudge filters, but it uses OpenSSL out of the box rather than GPG. A different workflow, and there are certainly pros and cons to both ways. [1] https://github.com/elasticdog/transcrypt https://github.com/elasticdog/transcrypt
kassemi 12y ago
I find the teams I work with generally end up transmitting secrets over untrusted centralized chat protocols. I'm glad to see folks stepping into this space and popularizing and simplifying this sort of security! For environment-based configuration values and multi-member teams, I wrote conf_conf [1]. [1] https://github.com/jkassemi/conf_conf https://github.com/jkassemi/conf_conf
- elasticdog 12y ago
  I've slowly been able to train my coworkers to use https://ezcrypt.it/ https://ezcrypt.it/ when sharing secrets rather than putting them directly into untrusted chat. Works great.
defunct 12y ago
I have been looking for a Python solution like this that does not encrypt the entire file, only the actual password entries. I stumbled upon Jaeger a while back which is written in Go. https://github.com/jyap808/jaeger https://github.com/jyap808/jaeger By encrypting only the password entries, version control systems can still track plain text configuration changes without showing secrets. Does anyone know if something like Jaeger is around but written in Python?
nchelluri 12y ago
Vaguely along the same lines, I use something similar for keeping my journal. Because I'm paranoid. :P I create/view/edit files and then I push the directory containing them to some git remote. https://gist.github.com/anonymous/df1ba398dd4e4748c6cd https://gist.github.com/anonymous/df1ba398dd4e4748c6cd Usage commands at the bottom.
mafro 12y ago
Shameless plug for a very simple tar-like CLI app done in Python: https://github.com/mafrosis/sesame https://github.com/mafrosis/sesame
crdoconnor 12y ago
Ansible handles this issue very cleanly with a feature called "vault": http://docs.ansible.com/playbooks_vault.html http://docs.ansible.com/playbooks_vault.html I think it probably works better integrated into the deployment system. The developer can still write {{ DBPASSWORD }} wherever they need and not have to worry that they don't know what the password on production or staging is.
- YesThatTom2 12y ago
  The README.md explains how to do this cleanly with Puppet and Hiera.
dmourati 12y ago
We used something similar for Puppet: https://github.com/TomPoulton/hiera-eyaml https://github.com/TomPoulton/hiera-eyaml Really neat thing was that you could inline encrypt just the secretes making the rest of the Puppet module viewable and comprehensible.
- Terretta 12y ago
  Looks like the read me mentions this for strings.
perlancar 12y ago
Personally, I don't use anything. I just check in my .gpg files and open with Emacs, which will decrypt/reencrypt the files for me. Any other editor worth their salt should have similar functionality.
- YesThatTom2 12y ago
  Good to know! This was originally written for Puppet files. The Puppet Master needs to have a separate key that permits it to decrypt all the files.
A1kmm 12y ago
If you can sneak a key into keyrings/live/pubring.gpg, and wait until someone runs blackbox_update_all_files without noticing, you then have access to all files. Perhaps it should keep a signed copy of the last version of pubring and show a some kind of diff between the verified previous version and the current version before going ahead.
peterwwillis 12y ago
I think this tool is useful, but it could be even more useful if the extra manual steps were automated as part of the tool. I can see myself constantly looking for examples to perform each step, rather than just using an intuitive tool.
pjjw 12y ago
additional close system: https://github.com/jschauma/jass https://github.com/jschauma/jass this solves the large team problem, too- it fetches membership data and ssh pubkeys from ldap. great if you have it, but only exists in large organizations. this of course doesn't do the gitwork for you, but it does avoid creation of another set of keys to be mismanaged.

150 points by akerl_ 12y ago

Interesting. For storing passwords I use Pass[1] which encrypts everything with GPG and then store it using Git in a private bitbucket repo. Works really well for me. I love the simplicity and ease of use. [1]http://www.zx2c4.com/projects/password-store/ http://www.zx2c4.com/projects/password-store/

akerl_ 12y ago
One of the things that looked cool about blackbox to me was the support for encrypting the data with multiple keys and managing that. I know some folks who use Pass and recommend it highly, but it seems focused on the personal use case like what you're doing. I do still need to set up Pass myself, and was thinking about making an Alfred workflow to let me quickly access credentials. Does such a thing already exist? (searching for "pass" is not exactly effective :))
- dnerdy 12y ago
  You can specify multiple gpg ids when using "pass init": pass init [-p sub-folder] user1@example.com user2@example.com This enables you to control access per subfolder. You can keep your personal passwords in one subfolder and then have various shared subfolders. "pass mv" and "pass cp" will reencrypt with the gpg ids for each subfolder as necessary. You can also "pass init" an existing subfolder to add or remove gpg ids for existing files. See the man page for details [1]. [1] http://git.zx2c4.com/password-store/about/ http://git.zx2c4.com/password-store/about/

scosman 12y ago

why not just use openssl or another command line tool? To encode: openssl aes-256-cbc -a -salt -in secret.zip -out secret.zip.enc To decode: openssl aes-256-cbc -d -a -in secret.zip.enc -out secret2.zip

scosman 12y ago
and nevermind. The doc explain why (large teams, key sharing, etc)
elasticdog 12y ago
That's essentially what I've automated with transcrypt (which uses Git's clean/smudge filters) to do that for you transparently: https://github.com/elasticdog/transcrypt https://github.com/elasticdog/transcrypt
YesThatTom2 12y ago
Why not? Why would you want to? Computers exist to make things easier, not harder.
e12e 12y ago
Maybe when openssl supports gcm or some other authenticated symmetric cipher mode on the command line?

deadponies 12y ago

Storing anything of value in the cloud, regardless of encryption, is the very essence of stupidity. How many times does this fact need to be reiterated in the news before people get it.

orblivion 12y ago
When was it ever in the news that something encrypted was compromised?
- therealmarv 12y ago
  http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryption-codes-security http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryp...
  - orblivion 12y ago
    I remember this story, though I forgot exactly how deep it went. I thought it was just compromising certificate authorities, or sneaking in physically to catch part of the stream that's not encrypted like they did with Google. That's irrelevant here because the only computers that would hold decrypted info in the case of Blackbox are those computers which would anyway hold it decrypted without Blackbox. So there's still the part about compromising the encryption software. Well, GPG is not proprietary so that's a lot harder. As far as somehow brute forcing the cyphertext, I guess that may be possible, though they don't give clear details about that here. That being the case, they still don't say anything about GPG. Perhaps the biggest argument of all is that Edward Snowden himself uses it, knowing everything he knows.
Sir_Cmpwn 12y ago
You could write properly encrypted data on a billboard in Times Square and it would still be secure.
aftbit 12y ago
A few years ago, my company was setting up a second office a few hours drive away. They wanted a replicated file share between the two offices. For the initial sync, they drove a NAS from the main office to the satellite. Then they had (openvpn-encrypted) rsyncs on a nightly basis. But the ITAR directory on the NAS was never rsynced. Instead, they drove the files out there every week using a flash drive. Seemed like a crazy policy to me, but after Heartbleed, who knows?

deleted 12y ago

[deleted]

yaccz 12y ago

1. store secretes in separate repo from code itself. Problem solved. 2. You still need to manually decrupt and encrypt files when you want to work with them. 3. It's written in shell. I'll never trust shell programs to be stable. They are too fragile to external changes and will break in runtime. I'm never ever work with secrets with shell wrapper.

vanilla 12y ago

git-crypt[1] also does per file encryption [1]: https://www.agwa.name/projects/git-crypt/ https://www.agwa.name/projects/git-crypt/

agwa 12y ago
And the upcoming version (which is overdue--I've been busy) will support encrypting with GPG with multiple recipients, like BlackBox.

miles_matthias 12y ago

we use https://github.com/ahoward/sekrets https://github.com/ahoward/sekrets at dojo4. love it.

jordigh 12y ago

Why does the HN title say "in git"? This tool seems to be specifically built to handle hg as well. A pleasant surprise for hg users like me. :-)

akerl_ 12y ago
My apologies; I took the title from the name/tagline of the repo. It won't let me edit the title now, but if a mod wanted to do me the favor and make it say "git/mercurial"...
- jordigh 12y ago
  Oh, I didn't notice the tagline. HN favours the original article's title in most cases, so I suppose a tagline is about as close to a title as we have in this case.
  - YesThatTom2 12y ago
    I've corrected the tag line. Thanks for pointing that out! Ironically Blackbox was originally for a Puppet repo that used Mercurial and only supported Git as a skunkswork project. Now it is devoid of any Puppet dependencies and is mostly used for Git, but the Mercurial support is solid. In fact, you could easily extend it to support other systems. CVS support, anyone?

rphillips 12y ago

Interesting... Very similar to my golang project. https://github.com/rphillips/gosec https://github.com/rphillips/gosec

elasticdog 12y ago

There are a lot of alternatives in this space (I wrote transcrypt [1] for just this purpose), but I think the killer features here are that you don't need shared credentials since you can have multiple keys, and that it's generic enough to work with multiple VCS's. With transcrypt, I do the encryption/decryption transparently once a repository has been configured by utilizing Git's clean/smudge filters, but it uses OpenSSL out of the box rather than GPG. A different workflow, and there are certainly pros and cons to both ways. [1] https://github.com/elasticdog/transcrypt https://github.com/elasticdog/transcrypt

kassemi 12y ago

I find the teams I work with generally end up transmitting secrets over untrusted centralized chat protocols. I'm glad to see folks stepping into this space and popularizing and simplifying this sort of security! For environment-based configuration values and multi-member teams, I wrote conf_conf [1]. [1] https://github.com/jkassemi/conf_conf https://github.com/jkassemi/conf_conf

elasticdog 12y ago
I've slowly been able to train my coworkers to use https://ezcrypt.it/ https://ezcrypt.it/ when sharing secrets rather than putting them directly into untrusted chat. Works great.

defunct 12y ago

I have been looking for a Python solution like this that does not encrypt the entire file, only the actual password entries. I stumbled upon Jaeger a while back which is written in Go. https://github.com/jyap808/jaeger https://github.com/jyap808/jaeger By encrypting only the password entries, version control systems can still track plain text configuration changes without showing secrets. Does anyone know if something like Jaeger is around but written in Python?

nchelluri 12y ago

Vaguely along the same lines, I use something similar for keeping my journal. Because I'm paranoid. :P I create/view/edit files and then I push the directory containing them to some git remote. https://gist.github.com/anonymous/df1ba398dd4e4748c6cd https://gist.github.com/anonymous/df1ba398dd4e4748c6cd Usage commands at the bottom.

mafro 12y ago

Shameless plug for a very simple tar-like CLI app done in Python: https://github.com/mafrosis/sesame https://github.com/mafrosis/sesame

crdoconnor 12y ago

Ansible handles this issue very cleanly with a feature called "vault": http://docs.ansible.com/playbooks_vault.html http://docs.ansible.com/playbooks_vault.html I think it probably works better integrated into the deployment system. The developer can still write {{ DBPASSWORD }} wherever they need and not have to worry that they don't know what the password on production or staging is.

YesThatTom2 12y ago
The README.md explains how to do this cleanly with Puppet and Hiera.

dmourati 12y ago

We used something similar for Puppet: https://github.com/TomPoulton/hiera-eyaml https://github.com/TomPoulton/hiera-eyaml Really neat thing was that you could inline encrypt just the secretes making the rest of the Puppet module viewable and comprehensible.

Terretta 12y ago
Looks like the read me mentions this for strings.

perlancar 12y ago

Personally, I don't use anything. I just check in my .gpg files and open with Emacs, which will decrypt/reencrypt the files for me. Any other editor worth their salt should have similar functionality.

YesThatTom2 12y ago
Good to know! This was originally written for Puppet files. The Puppet Master needs to have a separate key that permits it to decrypt all the files.

A1kmm 12y ago

If you can sneak a key into keyrings/live/pubring.gpg, and wait until someone runs blackbox_update_all_files without noticing, you then have access to all files. Perhaps it should keep a signed copy of the last version of pubring and show a some kind of diff between the verified previous version and the current version before going ahead.

peterwwillis 12y ago

I think this tool is useful, but it could be even more useful if the extra manual steps were automated as part of the tool. I can see myself constantly looking for examples to perform each step, rather than just using an intuitive tool.

pjjw 12y ago

additional close system: https://github.com/jschauma/jass https://github.com/jschauma/jass this solves the large team problem, too- it fetches membership data and ssh pubkeys from ldap. great if you have it, but only exists in large organizations. this of course doesn't do the gitwork for you, but it does avoid creation of another set of keys to be mismanaged.