3 ms·

It seems to me that cost is negligible for any of the use cases. The KMS CMK cost will dominate access cost in my estimation. I see that Credstash cites latency

by ak217 10y ago

It seems to me that cost is negligible for any of the use cases. The KMS CMK cost will dominate access cost in my estimation. I see that Credstash cites latency as a factor in favor of S3, which is not a major factor if you retrieve a secret only once in a while and put it into memory, but could enable cool use cases where you really retrieve it on demand. S3 does have other advantages - ease of use and granularity of IAM access control, I think. The Credstash documentation and code lead me to believe you are responsible for managing your own IAM policies - it won't set the policies in credential writer mode. Is that not correct? I expect this functionality to be part of the secret manager.

eropple 10y ago
> I see that Credstash cites latency as a factor in favor of S3, which is not a major factor if you retrieve a secret only once in a while and put it into memory Sorry, I wasn't clear. I meant "cheaper" in terms of time. I have a lot of machines in flight at any time, and I do actively roll credentials--chef-zero runs every five minutes and pulls credentials out of Credstash. S3 accesses add latency, which slows my converges (sometimes by a lot; I currently have around ~25 secret lookups in one particular Chef run). > I expect this functionality to be part of the secret manager. I'm not saying you're wrong to want that, but I'm very skeptical of it--extremely skeptical. I can't envision a situation where I'd rather have policy management as part of the secret manager as opposed to part of my provisioning infrastructure. My KMS accesses are in IAM roles; there is no situation in which I would ever want Credstash to manage those, because I have other stuff in them! I build my policies in CloudFormation, just like I do everything else. If anything, my biggest criticism of Credstash is that it has a hardcoded table name that I'd rather was an environment variable so that I could establish that via CloudFormation and dump it into an env var via Chef.
- ak217 10y ago
  In many cases, I think the secret manager should be part of your provisioning infrastructure. There are certainly other reasonable ways to structure this though - as long as you're clear on your threat model. Would you agree that the secret manager should at least print out a suggested IAM policy for you to use? IAM policies are complex, tricky, and easy to mess up. I think it should absolutely be the job of the secret manager to at least sanity check your policies, if not write them directly. > S3 accesses add latency [vs dynamodb], which slows my converges That's useful to know, thanks.
  - eropple 10y ago
    I don't find IAM policies to be tricky, to be honest. It may just be familiarity; I have been doing this for a little bit of time now. I'm also not really sure what sort of interface for generating those policies without introducing to the secret manager its own notion of "policies"--at which point you're duplicating effort and work. (I also wouldn't be able to use any policy spat out by the tool in the first place--I use a Cfer, Ruby DSL, to generate CloudFormation and all I'd be doing is transliterating it over.)
    - ak217 10y ago
      > I don't find IAM policies to be tricky, to be honest. It may just be familiarity; I have been doing this for a little bit of time now. Then you are an elite AWS operator, but you also have to think about how to teach and review work from other, less skilled operators working with you. In my experience it's not easy at all to do that with something as critical as IAM, so when I'm orchestrating resources, I heavily rely on IAM policy automation and built-in sanity checks.
      - eropple 10y ago
        My policies are automated. I wrote the automation. :) And thank you for the compliment, but I wouldn't call myself "elite", merely "pretty good". Seriously, this is literally all it is: def kms_read_policy(contexts) contexts.map do |context| { PolicyName: "kms-read", PolicyDocument: { Version: '2012-10-17', Statement: [ { Effect: 'Allow', Action: 'kms:Decrypt' , Resource: [ Cfer.cfize('arn:aws:kms:C{AWS.region}:' \ 'C{AWS.account_id}:key/C{Fn.ref(:CredstashKey)}') ], Condition: { StringEquals: context.map do |k, v| [ "kms:EncryptionContext:#{k}", v ] end.to_h } } ] } } end.to_a end And a single DynamoDB policy in a managed policy that allows for querying the `credential-store` table. This just is not rocket science. It's helped by cfer[0] and cfer-provisioning[1] being pretty great, but it's very achievable. [0] - https://github.com/seanedwards/cfer https://github.com/seanedwards/cfer [1] - https://github.com/seanedwards/cfer-provisioning https://github.com/seanedwards/cfer-provisioning (I'm a maintainer)

by ak217 10y ago

eropple 10y ago

> I see that Credstash cites latency as a factor in favor of S3, which is not a major factor if you retrieve a secret only once in a while and put it into memory Sorry, I wasn't clear. I meant "cheaper" in terms of time. I have a lot of machines in flight at any time, and I do actively roll credentials--chef-zero runs every five minutes and pulls credentials out of Credstash. S3 accesses add latency, which slows my converges (sometimes by a lot; I currently have around ~25 secret lookups in one particular Chef run). > I expect this functionality to be part of the secret manager. I'm not saying you're wrong to want that, but I'm very skeptical of it--extremely skeptical. I can't envision a situation where I'd rather have policy management as part of the secret manager as opposed to part of my provisioning infrastructure. My KMS accesses are in IAM roles; there is no situation in which I would ever want Credstash to manage those, because I have other stuff in them! I build my policies in CloudFormation, just like I do everything else. If anything, my biggest criticism of Credstash is that it has a hardcoded table name that I'd rather was an environment variable so that I could establish that via CloudFormation and dump it into an env var via Chef.

ak217 10y ago
In many cases, I think the secret manager should be part of your provisioning infrastructure. There are certainly other reasonable ways to structure this though - as long as you're clear on your threat model. Would you agree that the secret manager should at least print out a suggested IAM policy for you to use? IAM policies are complex, tricky, and easy to mess up. I think it should absolutely be the job of the secret manager to at least sanity check your policies, if not write them directly. > S3 accesses add latency [vs dynamodb], which slows my converges That's useful to know, thanks.
- eropple 10y ago
  I don't find IAM policies to be tricky, to be honest. It may just be familiarity; I have been doing this for a little bit of time now. I'm also not really sure what sort of interface for generating those policies without introducing to the secret manager its own notion of "policies"--at which point you're duplicating effort and work. (I also wouldn't be able to use any policy spat out by the tool in the first place--I use a Cfer, Ruby DSL, to generate CloudFormation and all I'd be doing is transliterating it over.)
  - ak217 10y ago
    > I don't find IAM policies to be tricky, to be honest. It may just be familiarity; I have been doing this for a little bit of time now. Then you are an elite AWS operator, but you also have to think about how to teach and review work from other, less skilled operators working with you. In my experience it's not easy at all to do that with something as critical as IAM, so when I'm orchestrating resources, I heavily rely on IAM policy automation and built-in sanity checks.
    - eropple 10y ago
      My policies are automated. I wrote the automation. :) And thank you for the compliment, but I wouldn't call myself "elite", merely "pretty good". Seriously, this is literally all it is: def kms_read_policy(contexts) contexts.map do |context| { PolicyName: "kms-read", PolicyDocument: { Version: '2012-10-17', Statement: [ { Effect: 'Allow', Action: 'kms:Decrypt' , Resource: [ Cfer.cfize('arn:aws:kms:C{AWS.region}:' \ 'C{AWS.account_id}:key/C{Fn.ref(:CredstashKey)}') ], Condition: { StringEquals: context.map do |k, v| [ "kms:EncryptionContext:#{k}", v ] end.to_h } } ] } } end.to_a end And a single DynamoDB policy in a managed policy that allows for querying the `credential-store` table. This just is not rocket science. It's helped by cfer[0] and cfer-provisioning[1] being pretty great, but it's very achievable. [0] - https://github.com/seanedwards/cfer https://github.com/seanedwards/cfer [1] - https://github.com/seanedwards/cfer-provisioning https://github.com/seanedwards/cfer-provisioning (I'm a maintainer)