7 ms·

Simple and flexible tool for managing secrets

139 points by known 6y ago

varbhat 6y ago
Also see https://www.gopass.pw/ https://www.gopass.pw/
- lucideer 6y ago
  I don't think this is the same thing. Gopass is a fully featured user password manager, SOPS is a developer library for encrypting strings in files. I guess one could use Mozilla SOPS to create something like gopass...
AnonHP 6y ago
To the maintainer of this repo, please make it accessible and easy to understand. The animated image is a bit too fast to read and follow, especially considering that there is absolutely no explanation right at the beginning for what the tool does AND why the tool is required for anyone. I had to scroll down a lot to get to the part where the important details are (which could answer the question "why should I even look at or consider this tool?").
- acqq 6y ago
  Yes, only based on the animated image I guess that the goal of the tool could be to encrypt just the passwords which are expected to be present in some plain text files in some bigger setups. It would help to have some explanation of what the goals and use scenarios are, and general assumptions.
lucideer 6y ago
This seems like a much easier-to-apply, automatic version of Ansible Vault's YAML variable encryption[0]. Personally I've never needed the extra per-YAML-entry control Ansible Vault gives you, so this seems a better approach. [0] https://docs.ansible.com/ansible/latest/user_guide/vault.html#id13 https://docs.ansible.com/ansible/latest/user_guide/vault.htm...
- thinkmassive 6y ago
  Yes, and you can easily start using SOPS for Ansible secrets via the community.sops collection: https://github.com/ansible-collections/community.sops https://github.com/ansible-collections/community.sops
ithkuil 6y ago
If you're still interested in something similar tailored for kubernetes, take a look at https://github.com/bitnami-labs/sealed-secrets https://github.com/bitnami-labs/sealed-secrets
- indale 6y ago
  If you like the configuration of SOPS but you want to use bitnamis sealed secrets as a K8s native approach feel free to have a look at my alternative cli https://github.com/dschniepp/sealit https://github.com/dschniepp/sealit
guessmyname 6y ago
I am surprised Mozilla developers chose to write this project in Go instead of Rust, but do not get me wrong, I like Go, it is my primary programming language at work, and I always try to advocate for the right tool for the job. I still find it curious, though. I guess it was easier to translate Python to Go than to Rust.
- sofixa 6y ago
  And a tool like that doesn't get any advantage from Rust - manual memory management would just be a chore with no benefits, and Go's crypto library is pretty good ( no idea about Rust's)
- tclancy 6y ago
  I’m guessing it’s the deployment story that made them choose Go.
FiloSottile 6y ago
The develop branch has merged support for age, too! https://github.com/mozilla/sops/tree/develop https://github.com/mozilla/sops/tree/develop
- github_drop_ice 6y ago
  Would really, really love to get an official release of sops with age support. Thanks for the tool--compared to GPG age is a complete breath of fresh air.
YesThatTom2 6y ago
Neat! I like that it maintains everything in the same file. No need to mess with .gitignore Have you tried BlackBox? Projects like https://github.com/StackExchange/blackbox https://github.com/StackExchange/blackbox have the benefit of using GPG group encryption. Each person has their own secret key but any one key can decrypt the file. This makes key rotation easier.
- willejs 6y ago
  Sops can use GPG key groups too. There are multiple encryption methods available, such as GPG, KMS (GCP and AWS) and azure key vault.
10yrengineer 6y ago
I didn't understand where keys are stored. Isn't that still an entry point?
- thinkmassive 6y ago
  Keys are stored in a PGP keyring and/or cloud Key Management Service (KMS). This is mentioned in the opening statement of the readme, and usage is detailed in section 2 of the docs: https://github.com/mozilla/sops#usage https://github.com/mozilla/sops#usage
avh02 6y ago
worked with this at a previous job - pro tip if you're doing anything with KMS (and are doing a lot of role assumption, etc), set AWS_SDK_LOAD_CONFIG=1 https://github.com/mozilla/sops/issues/471#issuecomment-503670652 https://github.com/mozilla/sops/issues/471#issuecomment-5036... (and i'm sure countless other references) Generally found that setting that env var with anything mildly complicated (in go!) in terms of AWS roles helps. Other libraries/clis/etc (in other languages) tend to transparently traverse the credentials chain for you.
bogwog 6y ago
What is the purpose of encrypting just the values and not the keys? If you need to run the file through sops to decrypt the values before using it, why not just encrypt the entire file? Is it just for easier debugging, so you can see the structure of the encrypted file?
- ibrarmalik 6y ago
  For version controlling this could be useful.
Boxxed 6y ago
Not to pile-on, but I've used envkey in the past and would definitely recommend it to anyone: https://www.envkey.com/ https://www.envkey.com/
edf13 6y ago
A quick look at the repo and it doesn't shout simple and flexible...
- jeffrallen 6y ago
  When I review code related to security, I first start with the dependency list. I don't keep looking if they depend on 50 external packages (wc -l go.mod).
jlgaddis 6y ago
Even better, IMO, would be if all targets were also proxies and a client could choose -- at "query time" -- any combination of (proxy, target) that they prefer. If you wanted to go a step further, you can even allow "chaining" of proxies, such that the path a query takes might be, in an extreme example, similar to how Tor operates: Client -> Proxy 1 -> Proxy 2 -> Proxy 3 -> Target -> Resolver -- Anyways, this is kinda sorta interesting, I guess, but honestly I'm more excited by and looking forward to the (hopefully!) eventual adoption and roll-out of "DNS SVCB and HTTPS RRs" [0] -- one of the other I-Ds (linked in the OP) on which ODoH is built -- and I suspect many other HN'ers will be as well (although I'd happily settle for SRV RR support in browsers). -- [0]: https://tools.ietf.org/html/draft-ietf-dnsop-svcb-https-02 https://tools.ietf.org/html/draft-ietf-dnsop-svcb-https-02
- sciurus 6y ago
  Did you comment on the wrong post? This seems unrelated to sops.
  - jlgaddis 6y ago
    Thanks, I sure did. I've copied it to the "Cloudflare and Apple design a new privacy-friendly internet protocol" [0] thread that it was intended for, but it's too late to delete it from this one. -- [0]: https://news.ycombinator.com/item?id=25344358 https://news.ycombinator.com/item?id=25344358
reidjs 6y ago
Another tool I like using for server secrets is syncthing https://syncthing.net/ https://syncthing.net/
gingerlime 6y ago
Throwing another tiny open-source option into the arena: Envwarden[0] is a simple wrapper for managing server secrets with the Bitwarden[1] password manager. Not affiliated with Bitwarden in any way. [0] https://github.com/envwarden/envwarden https://github.com/envwarden/envwarden [1] https://bitwarden.com/ https://bitwarden.com/
thinkmassive 6y ago
Anyone saving encrypted secrets in git will probably find it helpful to enable cleartext diffs: https://github.com/mozilla/sops#48showing-diffs-in-cleartext-in-git https://github.com/mozilla/sops#48showing-diffs-in-cleartext... If you have many secrets you'll probably still end up breaking them out into different files to minimize conflicts between multiple in-progress branches. A good place to start is keeping a secrets file per environment+service: - sops-dev-mywebapp.yml - sops-dev-mydb.yml - sops-prod-mywebapp.yml - sops-prod-mydb.yml
frompdx 6y ago
For those using Terragrunt (Terraform wrapper) there is a built in function for decrypting files encrypted with SOPS. https://terragrunt.gruntwork.io/docs/reference/built-in-functions/#sops_decrypt_file https://terragrunt.gruntwork.io/docs/reference/built-in-func...
nonninz 6y ago
Wow this thread made me realize there are so many tools for this. Just to throw in another one, at my company we are using git-crypt [1], and it works pretty well in our case [1] https://github.com/AGWA/git-crypt https://github.com/AGWA/git-crypt
- tinalumfoil 6y ago
  My experience with git-crypt is it works fine until you have a merge conflict. This tool seems a lot more compelling in that regard, since only encrypting the subset of the file that needs to be secret I'd imagine diffs and merge conflict would be pretty seamless.
ComputerGuru 6y ago
We developed an open standard for storing encrypted secrets alongside code in your git/whatever repo: https://neosmart.net/blog/2020/securestore-open-secrets-format/ https://neosmart.net/blog/2020/securestore-open-secrets-form... The format is language/platform agnostic and we’ve releases libraries for .NET/C# and rust, as well as developed two separate CLI utilities that can create and manage the stores.

139 points by known 6y ago

varbhat 6y ago

Also see https://www.gopass.pw/ https://www.gopass.pw/

lucideer 6y ago
I don't think this is the same thing. Gopass is a fully featured user password manager, SOPS is a developer library for encrypting strings in files. I guess one could use Mozilla SOPS to create something like gopass...

AnonHP 6y ago

To the maintainer of this repo, please make it accessible and easy to understand. The animated image is a bit too fast to read and follow, especially considering that there is absolutely no explanation right at the beginning for what the tool does AND why the tool is required for anyone. I had to scroll down a lot to get to the part where the important details are (which could answer the question "why should I even look at or consider this tool?").

acqq 6y ago
Yes, only based on the animated image I guess that the goal of the tool could be to encrypt just the passwords which are expected to be present in some plain text files in some bigger setups. It would help to have some explanation of what the goals and use scenarios are, and general assumptions.

lucideer 6y ago

This seems like a much easier-to-apply, automatic version of Ansible Vault's YAML variable encryption[0]. Personally I've never needed the extra per-YAML-entry control Ansible Vault gives you, so this seems a better approach. [0] https://docs.ansible.com/ansible/latest/user_guide/vault.html#id13 https://docs.ansible.com/ansible/latest/user_guide/vault.htm...

thinkmassive 6y ago
Yes, and you can easily start using SOPS for Ansible secrets via the community.sops collection: https://github.com/ansible-collections/community.sops https://github.com/ansible-collections/community.sops

ithkuil 6y ago

If you're still interested in something similar tailored for kubernetes, take a look at https://github.com/bitnami-labs/sealed-secrets https://github.com/bitnami-labs/sealed-secrets

indale 6y ago
If you like the configuration of SOPS but you want to use bitnamis sealed secrets as a K8s native approach feel free to have a look at my alternative cli https://github.com/dschniepp/sealit https://github.com/dschniepp/sealit

guessmyname 6y ago

I am surprised Mozilla developers chose to write this project in Go instead of Rust, but do not get me wrong, I like Go, it is my primary programming language at work, and I always try to advocate for the right tool for the job. I still find it curious, though. I guess it was easier to translate Python to Go than to Rust.

sofixa 6y ago
And a tool like that doesn't get any advantage from Rust - manual memory management would just be a chore with no benefits, and Go's crypto library is pretty good ( no idea about Rust's)
tclancy 6y ago
I’m guessing it’s the deployment story that made them choose Go.

FiloSottile 6y ago

The develop branch has merged support for age, too! https://github.com/mozilla/sops/tree/develop https://github.com/mozilla/sops/tree/develop

github_drop_ice 6y ago
Would really, really love to get an official release of sops with age support. Thanks for the tool--compared to GPG age is a complete breath of fresh air.

YesThatTom2 6y ago

Neat! I like that it maintains everything in the same file. No need to mess with .gitignore Have you tried BlackBox? Projects like https://github.com/StackExchange/blackbox https://github.com/StackExchange/blackbox have the benefit of using GPG group encryption. Each person has their own secret key but any one key can decrypt the file. This makes key rotation easier.

willejs 6y ago
Sops can use GPG key groups too. There are multiple encryption methods available, such as GPG, KMS (GCP and AWS) and azure key vault.

10yrengineer 6y ago

I didn't understand where keys are stored. Isn't that still an entry point?

thinkmassive 6y ago
Keys are stored in a PGP keyring and/or cloud Key Management Service (KMS). This is mentioned in the opening statement of the readme, and usage is detailed in section 2 of the docs: https://github.com/mozilla/sops#usage https://github.com/mozilla/sops#usage

avh02 6y ago

worked with this at a previous job - pro tip if you're doing anything with KMS (and are doing a lot of role assumption, etc), set AWS_SDK_LOAD_CONFIG=1 https://github.com/mozilla/sops/issues/471#issuecomment-503670652 https://github.com/mozilla/sops/issues/471#issuecomment-5036... (and i'm sure countless other references) Generally found that setting that env var with anything mildly complicated (in go!) in terms of AWS roles helps. Other libraries/clis/etc (in other languages) tend to transparently traverse the credentials chain for you.

bogwog 6y ago

What is the purpose of encrypting just the values and not the keys? If you need to run the file through sops to decrypt the values before using it, why not just encrypt the entire file? Is it just for easier debugging, so you can see the structure of the encrypted file?

ibrarmalik 6y ago
For version controlling this could be useful.

Boxxed 6y ago

Not to pile-on, but I've used envkey in the past and would definitely recommend it to anyone: https://www.envkey.com/ https://www.envkey.com/

edf13 6y ago

A quick look at the repo and it doesn't shout simple and flexible...

jeffrallen 6y ago
When I review code related to security, I first start with the dependency list. I don't keep looking if they depend on 50 external packages (wc -l go.mod).

jlgaddis 6y ago

Even better, IMO, would be if all targets were also proxies and a client could choose -- at "query time" -- any combination of (proxy, target) that they prefer. If you wanted to go a step further, you can even allow "chaining" of proxies, such that the path a query takes might be, in an extreme example, similar to how Tor operates: Client -> Proxy 1 -> Proxy 2 -> Proxy 3 -> Target -> Resolver -- Anyways, this is kinda sorta interesting, I guess, but honestly I'm more excited by and looking forward to the (hopefully!) eventual adoption and roll-out of "DNS SVCB and HTTPS RRs" [0] -- one of the other I-Ds (linked in the OP) on which ODoH is built -- and I suspect many other HN'ers will be as well (although I'd happily settle for SRV RR support in browsers). -- [0]: https://tools.ietf.org/html/draft-ietf-dnsop-svcb-https-02 https://tools.ietf.org/html/draft-ietf-dnsop-svcb-https-02

sciurus 6y ago
Did you comment on the wrong post? This seems unrelated to sops.
- jlgaddis 6y ago
  Thanks, I sure did. I've copied it to the "Cloudflare and Apple design a new privacy-friendly internet protocol" [0] thread that it was intended for, but it's too late to delete it from this one. -- [0]: https://news.ycombinator.com/item?id=25344358 https://news.ycombinator.com/item?id=25344358

reidjs 6y ago

Another tool I like using for server secrets is syncthing https://syncthing.net/ https://syncthing.net/

gingerlime 6y ago

Throwing another tiny open-source option into the arena: Envwarden[0] is a simple wrapper for managing server secrets with the Bitwarden[1] password manager. Not affiliated with Bitwarden in any way. [0] https://github.com/envwarden/envwarden https://github.com/envwarden/envwarden [1] https://bitwarden.com/ https://bitwarden.com/

thinkmassive 6y ago

Anyone saving encrypted secrets in git will probably find it helpful to enable cleartext diffs: https://github.com/mozilla/sops#48showing-diffs-in-cleartext-in-git https://github.com/mozilla/sops#48showing-diffs-in-cleartext... If you have many secrets you'll probably still end up breaking them out into different files to minimize conflicts between multiple in-progress branches. A good place to start is keeping a secrets file per environment+service: - sops-dev-mywebapp.yml - sops-dev-mydb.yml - sops-prod-mywebapp.yml - sops-prod-mydb.yml

frompdx 6y ago

For those using Terragrunt (Terraform wrapper) there is a built in function for decrypting files encrypted with SOPS. https://terragrunt.gruntwork.io/docs/reference/built-in-functions/#sops_decrypt_file https://terragrunt.gruntwork.io/docs/reference/built-in-func...

nonninz 6y ago

Wow this thread made me realize there are so many tools for this. Just to throw in another one, at my company we are using git-crypt [1], and it works pretty well in our case [1] https://github.com/AGWA/git-crypt https://github.com/AGWA/git-crypt

tinalumfoil 6y ago
My experience with git-crypt is it works fine until you have a merge conflict. This tool seems a lot more compelling in that regard, since only encrypting the subset of the file that needs to be secret I'd imagine diffs and merge conflict would be pretty seamless.

ComputerGuru 6y ago

We developed an open standard for storing encrypted secrets alongside code in your git/whatever repo: https://neosmart.net/blog/2020/securestore-open-secrets-format/ https://neosmart.net/blog/2020/securestore-open-secrets-form... The format is language/platform agnostic and we’ve releases libraries for .NET/C# and rust, as well as developed two separate CLI utilities that can create and manage the stores.