10 ms·

Ask HN: How is your company storing secrets in EC2?

• chrismeller 8y ago
  You’re confusing services and their intentions. KMS provides encryption. Yes it can store small values for you, but that’s it - encrypt this, decrypt that, rotate this key. Everything else is your job. This is the low level service. Secrets manager builds upon this. More access controls, some built in logic for rotation of secrets, etc. Which is correct for you depends upon what kind of secret you’re storing, what you do with it, how often and how you need to rotate or regenerate it... dozens of factors. Basically, KMS is the key master. It manages the keys to the data and will begrudgingly hang on to small bits of paper you’ve asked it to encrypt, but it does nothing else. Data in, encryption, data out, decryption. If an API key is one of those bits of data that’s fine. It knows not and cares not. Need to rotate it? Pull it out, decrypt it, rotate it, encrypt it, and shove it back in - all that is your business. Secrets Manager is the gate keeper. It needs the key master but it knows more about who and what is passing through and will help you out from time to time. Stored your most recent allegiance to Zuul but you haven’t reaffirmed it recently? He’ll be a bro and go ahead and send it back in for you and store the new allegiance until you need it again.
• laurentl 8y ago
  We’ve been using SSM parameter store to store our secrets (as secure strings). To manage access in a more granular manner we use a namespace approach where the parameters are prefixed with the service or API they’re attached to, or “global” if they’re meant for everyone. Each service has read rights for <service>-* and global-* parameters. We use a python library called SSM-cache to cache values locally and avoid the cost of a look-up each time. I’ve recently seen secret manager and I first thought it was a rebranding of SSM PS. A quick google search seems to indicate that this is more or less the case at the moment, but the service will probably pick up new features as it matures.
  • gtsteve 8y ago
    There is no additional charge for SSM parameter store but rate limits apply (and I can't find out what they are). I have read accounts from people on HN who work on large AWS accounts who have been bitten by this. The secret manager bills per request, so presumably that means they'll happily serve as many requests as you want. It also has some nice tooling around cycling secrets. It's an important distinction when choosing which one to work with. To contribute an additional idea not seen in this topic yet - my application predates both these services so it just reads from a JSON file in an s3 bucket that is encrypted at rest. It works just as well as all the above.
    • dmlittle 8y ago
      You should look into getParamtersByPath instead of getParameters. You should be able to get past the rate limits for quite some time (we haven't hit the getParamtersByPath limits yet)
• dmlittle 8y ago
  Similar to laurentl we use AWS SSM parameter store in combination with Chamber[1] to injects our secrets as environment variables to our application code. For services running on ECS, each task definition has an IAM Role attached that has permissions to use secrets stored in parameter store with a prefix unique for that service (each service encrypts its secrets with its own KMS key). For example, serviceA will use the prefix serviceA and have a secret under serviceA/DB_PASSWORD and serviceB will have its own secret under serviceB/DB_PASSWORD. Each service cannot access each others secrets and even if they could, they also need access to use the corresponding KMS key before they're able to read the secrets. Since secrets are added as environment variables we can inject them during startup with chamber. The entry point for your docker container might look as following `CMD ["chamber", "exec", "serviceA", "--", "./executable"]`. Chamber will inject the secrets store in SSM before starting the executable which reads the secrets from the env variables. For other services such as Lambda we do not use environment variables, instead we use the AWS SDK to fetch secrets during cold starts and cache them for the duration of the Lambda container. If you're using Node.js lambda functions this is a little bit trickier to do than Go or python lambdas but it's still possible. The Lambda function also has an IAM access to only fetch the secrets under its own prefix. For adding, removing or editing secrets we can use the chamber CLI with each individual devs AWS access keys. For example: `chamber write serviceA PASSWORD hunter2` to write a new secret or `chamber list serviceA` to list set secrets. [1] https://github.com/segmentio/chamber https://github.com/segmentio/chamber