4 ms·

I agree fully with you but I don't think secret management is as easy/cheap as some people pretend. On AWS, for example, each secret you store is $0.40 + $0.05

I agree fully with you but I don't think secret management is as easy/cheap as some people pretend. On AWS, for example, each secret you store is $0.40 + $0.05 per 10,000 API calls and that can add up if you only have 1 api key/password/etc per "Secret" (for an individual at least, I hate bleeding off $10+/mo to store tiny bits of strings). Then, once you have the secret stored, you need setup roles/policies to be able to retrieve them. I have this setup pretty well in my code now but getting there wasn't simple or easy from my perspective and keeping the list of secrets your IAM user can access up to date can be a pain as well. I'm working in a lambda environment so my options might be more limited but I'm interested to see how other people are solving this issue (maybe specifically for small/side projects). As it stands my lambdas all get a role applied to them that gives them access to the secrets but something not AWS-specific would need a "bootstrap secret" to be injected before the code could call out to the third-party to get the other secrets. For Lambdas I suppose I could inject that "bootstrap secret" in via environmental variables but now I've got a new issue to deal with. Injecting at build time via something like GitHub Actions Secrets is an option I guess. All that to say, while I agree secrets should never be in source, in practice it's not super easy (I'd love to be proven wrong, maybe I'm not doing it right).

jeremymcanally 4y ago
We use Doppler and some other custom built tooling, and it works really well. The Doppler pricing is really fair IMO, and their tooling adds a lot of value for us so I'm OK with paying for it. Software security is rarely free (even with an OSS tool you've got infra and management costs), but the cost is almost always cheaper than a major breach that could stem from something like this incident, which fortunately was pretty contained.
danenania 4y ago
You could check out https://www.envkey.com https://www.envkey.com (disclaimer: I'm the founder). Our goal is to simplify this as much as possible. We use client-side end-to-end encryption so you don't have to trust a third party, and pricing on our paid plans is based on number of users, not number of secrets.
- philsnow 4y ago
  Having helped build the production secret management bits at Airbnb [0], I would encourage most people to buy, not build. It was a large task and I'm 95% sure we didn't add more value than the cost of all the people working on it for as long as it took, compared to buying a solution. Just like the rest of Airbnb started before VPCs were GA and thus required a large engineering investment to move everything to VPCs, we started on the secret management stuff before there were a lot of good other options available (though arguably Hashicorp Vault was around and mature enough at the time, and would have been the best alternative). I haven't looked at envkey for production use but I've definitely considered it for home use since it's just so deliciously simple. [0] https://medium.com/airbnb-engineering/production-secret-management-at-airbnb-ad230e1bc0f6 https://medium.com/airbnb-engineering/production-secret-mana...
- eins1234 4y ago
  I was looking at this with great interest, until I looked at the pricing and found out the lowest hosted pricing tier starts at $150/month. I know there's a free "community" hosted version, but I'm not sure what the differences is outside of the limits and support, and I'd prefer to see the pricing scale up a bit more gently than 0 -> $150 as soon as I reach the limits of the free offering.
  - danenania 4y ago
    Perhaps you'll see this if you get notified of replies: per your suggestion, we have now introduced a lower tier in between the free tier and the ~150/month tier.
  - eins1234 4y ago
    Also, the list of limitations on the self-hosted open-source version reads like something that is only offered to tick an open-source checkbox: https://docs-v2.envkey.com/docs/self-hosting-open-source-envkey https://docs-v2.envkey.com/docs/self-hosting-open-source-env... Very clearly cut down to the point where it's not feasible for anything outside of toy projects.
    - 4y ago
capableweb 4y ago
> I agree fully with you but I don't think secret management is as easy/cheap as some people pretend. On AWS, for example, each secret you store is $0.40 + $0.05 per 10,000 API calls and that can add up if you only have 1 api key/password/etc per "Secret" (for an individual at least, I hate bleeding off $10+/mo to store tiny bits of strings). Then, once you have the secret stored, you need setup roles/policies to be able to retrieve them. There is a middle-step between "lets have API tokens committed in SCM" and "lets deploy a full-authentication system/use this costly solution", and that is using environment variables. In your code, do `process.env.MY_SECRET_KEY` instead of `myGitHubPersonalToken` and then when you run the program, run it with ` MY_SECRET_KEY=myGitHubPersonalToken npm start`. Magically, you can commit your code without exposing any secrets, and share the secret where you need it out-of-band. Zero-cost, actually easier to configure your software when you need it, and as a bonus, people won't get access to your infrastructure in case someone gets a hold of your source code. That npm inc isn't aware (or failed to uphold the code quality) of environment variables for secrets is embarrassing.
- tmp538394722 4y ago
  > when you run the program, run it with ` MY_SECRET_KEY=myGitHubPersonalToken npm start` But where does this live? Or do you literally mean that Jane The Sys Admin is supposed to type this into her terminal every time the service restarts in the middle of the night? What if I need to replace a node? Or scale a service? How do these secrets get there?
  - capableweb 4y ago
    > But where does this live? Or do you literally mean that Jane The Sys Admin is supposed to type this into her terminal every time the service restarts in the middle of the night? Depends on how the service is deployed. If you're just running it on a Digital Ocean instance by manually SSHing into the instance and running systemd services, define it in the .service file (it supports defining environment variables). If you're doing instances via automation (like Terraform), most of them (including Terraform) supports loading things from environment variables. So you run `MY_SECRET_KEY=myGitHubPersonalToken terraform apply` when you create the instance, and use the environment variable in your hcl definitions.
scarface74 4y ago
Don’t use Secret Manager then. You can use SSM Parameter Store with a type of “secret string”. It’s much cheaper.

jeremymcanally 4y ago

We use Doppler and some other custom built tooling, and it works really well. The Doppler pricing is really fair IMO, and their tooling adds a lot of value for us so I'm OK with paying for it. Software security is rarely free (even with an OSS tool you've got infra and management costs), but the cost is almost always cheaper than a major breach that could stem from something like this incident, which fortunately was pretty contained.

danenania 4y ago

You could check out https://www.envkey.com https://www.envkey.com (disclaimer: I'm the founder). Our goal is to simplify this as much as possible. We use client-side end-to-end encryption so you don't have to trust a third party, and pricing on our paid plans is based on number of users, not number of secrets.

philsnow 4y ago
Having helped build the production secret management bits at Airbnb [0], I would encourage most people to buy, not build. It was a large task and I'm 95% sure we didn't add more value than the cost of all the people working on it for as long as it took, compared to buying a solution. Just like the rest of Airbnb started before VPCs were GA and thus required a large engineering investment to move everything to VPCs, we started on the secret management stuff before there were a lot of good other options available (though arguably Hashicorp Vault was around and mature enough at the time, and would have been the best alternative). I haven't looked at envkey for production use but I've definitely considered it for home use since it's just so deliciously simple. [0] https://medium.com/airbnb-engineering/production-secret-management-at-airbnb-ad230e1bc0f6 https://medium.com/airbnb-engineering/production-secret-mana...
eins1234 4y ago
I was looking at this with great interest, until I looked at the pricing and found out the lowest hosted pricing tier starts at $150/month. I know there's a free "community" hosted version, but I'm not sure what the differences is outside of the limits and support, and I'd prefer to see the pricing scale up a bit more gently than 0 -> $150 as soon as I reach the limits of the free offering.
- danenania 4y ago
  Perhaps you'll see this if you get notified of replies: per your suggestion, we have now introduced a lower tier in between the free tier and the ~150/month tier.
- eins1234 4y ago
  Also, the list of limitations on the self-hosted open-source version reads like something that is only offered to tick an open-source checkbox: https://docs-v2.envkey.com/docs/self-hosting-open-source-envkey https://docs-v2.envkey.com/docs/self-hosting-open-source-env... Very clearly cut down to the point where it's not feasible for anything outside of toy projects.
  - 4y ago

capableweb 4y ago

> I agree fully with you but I don't think secret management is as easy/cheap as some people pretend. On AWS, for example, each secret you store is $0.40 + $0.05 per 10,000 API calls and that can add up if you only have 1 api key/password/etc per "Secret" (for an individual at least, I hate bleeding off $10+/mo to store tiny bits of strings). Then, once you have the secret stored, you need setup roles/policies to be able to retrieve them. There is a middle-step between "lets have API tokens committed in SCM" and "lets deploy a full-authentication system/use this costly solution", and that is using environment variables. In your code, do `process.env.MY_SECRET_KEY` instead of `myGitHubPersonalToken` and then when you run the program, run it with ` MY_SECRET_KEY=myGitHubPersonalToken npm start`. Magically, you can commit your code without exposing any secrets, and share the secret where you need it out-of-band. Zero-cost, actually easier to configure your software when you need it, and as a bonus, people won't get access to your infrastructure in case someone gets a hold of your source code. That npm inc isn't aware (or failed to uphold the code quality) of environment variables for secrets is embarrassing.

tmp538394722 4y ago
> when you run the program, run it with ` MY_SECRET_KEY=myGitHubPersonalToken npm start` But where does this live? Or do you literally mean that Jane The Sys Admin is supposed to type this into her terminal every time the service restarts in the middle of the night? What if I need to replace a node? Or scale a service? How do these secrets get there?
- capableweb 4y ago
  > But where does this live? Or do you literally mean that Jane The Sys Admin is supposed to type this into her terminal every time the service restarts in the middle of the night? Depends on how the service is deployed. If you're just running it on a Digital Ocean instance by manually SSHing into the instance and running systemd services, define it in the .service file (it supports defining environment variables). If you're doing instances via automation (like Terraform), most of them (including Terraform) supports loading things from environment variables. So you run `MY_SECRET_KEY=myGitHubPersonalToken terraform apply` when you create the instance, and use the environment variable in your hcl definitions.

scarface74 4y ago

Don’t use Secret Manager then. You can use SSM Parameter Store with a type of “secret string”. It’s much cheaper.