7 ms·

For me personally, it would be much more useful if I could play with the policies interactively - issue requests with some tagging (a special access key maybe?)

by krab 4y ago

For me personally, it would be much more useful if I could play with the policies interactively - issue requests with some tagging (a special access key maybe?) and see what permissions they need, what context they have and which of them are allowed/denied by the policies. And what if I edit this part, would it affect the recorded requests? At the moment, it usually takes me several iterations to tune the policy.

rollthehard6 4y ago
So much this, I don't understand why cloud providers don't offer an easy way to view this sort of thing as really it should all be in the logging anyway.
- lozenge 4y ago
  It seems like AWS isn't good with cross cutting concerns. Eg instead of Cloudformation being part of a service during development, that they could even use during testing the service, every team creates a custom api and cf is an afterthought. It wouldn't surprise me if every service is also evaluating against iam policies independently as well.
  - rollthehard6 4y ago
    That reminds me of the latter stages of Sun Microsystems, where it seemed like multiple teams were developing competing solutions for system management and patching. Seems a wasteful model and not one that is servicing customers.
  - grogenaut 4y ago
    Much of the services were built well before clod formation... Or cloud formation became popular... Or before everyone was willing to make the trade-off that features come later because they had to have cloud formation. Like any other best practice you invent along the way. I'll bet there's something about your 1/3/5/10/15 year old that isn't up to current snuff. People forget, AWS is massive and taking care of edge case after edge case while being in a bullet point war with other cloud providers.
    - erik_seaberg 4y ago
      Even new features sometimes don’t get CloudFormation support for quite a while (sometimes longer than third-party support like Terraform). I keep wondering why they didn’t want it for their own integration testing.
- cperciva 4y ago
  The execution of rulesets needs to be performed synchronously and as fast as possible, since requests are blocked until they pass the permission checks; but it doesn't need any sort of replication since each request is validated independently. Request logging can be performed asynchronously and has much more relaxed latency requirements, but data needs to be aggregated and replicated for durability. The extremely different requirements pretty much guarantee that these will be two completely separate systems; by Conway's Law, this means two completely separate teams. In practice, logging is more aimed at business analytics and billing. You're right that the information should be available, and there's no technical reason why it can't be made available, but there are technical reasons which influence the social reasons why it's not available.
- fulafel 4y ago
  AWS had the policy simulator for IAM policies but it hasn't been kept up to date wrt new features (like SCP)... Maybe they'll have something similar for this.
lockedinspace 4y ago
You can simulate a policy with AWS IAM Policy Simulator
- krab 4y ago
  This doesn't cut it. That tool helps if your role/user has a lot of policies that might interact between each other. But you already need to know the exact "actions" and context. On the other hand, I have a tool that calls some AWS services that may in turn call other AWS services. Now if something fails because of IAM denial, I have to go through the logs to figure out what it needed, sometimes it's not even in the logs (S3). Then add it to the policy and repeat to see the next failed call. I imagine being able to call real API calls, make them succeed and record what permissions it needed for each of those API calls. I don't need that as a permanent log, just as a development tool.
  - sickmate 4y ago
    You can do this with Localstack, however it's gated behind their pro service. https://docs.localstack.cloud/user-guide/aws/iam/#explainable-iam https://docs.localstack.cloud/user-guide/aws/iam/#explainabl...
  - kapilvt 4y ago
    All of the sdks support client side monitoring (CSM), so these sort of tools can be built client side. https://boto3.amazonaws.com/v1/documentation/api/1.10.46/guide/sdk-metrics.html#enable-sdk-metrics https://boto3.amazonaws.com/v1/documentation/api/1.10.46/gui... afaics the only challenge is mapping some of the apis to iam as its only 85% 1:1 There's also tools for helping with iam like (generator, and linter) https://github.com/salesforce/policy_sentry https://github.com/salesforce/policy_sentry https://github.com/duo-labs/parliament https://github.com/duo-labs/parliament
    - krab 4y ago
      > mapping some of the apis to iam as its only 85% 1:1 Yeah. Try to create an Elastic beanstalk app with only EB permissions.
ProjectMoonShot 4y ago
The book AWS Cookbook section 1.2 is on 'Generating a Least Privilege IAM Policy Based on Access Patterns'. [1] https://www.oreilly.com/library/view/aws-cookbook/9781492092599/ https://www.oreilly.com/library/view/aws-cookbook/9781492092... [2] https://docs.aws.amazon.com/IAM/latest/UserGuide/access-analyzer-policy-generation.html https://docs.aws.amazon.com/IAM/latest/UserGuide/access-anal...
- krab 4y ago
  This goes in the right direction. Still it's not 100% reliable (e.g. S3 isn't logged by default to cloudtrail, probably due to volume). Also, not really interactive. I'm talking mostly about ergonomics. Obviously we can solve the issue and create the right policy - but it's not as easy as it could be. That in turn leads to some people skipping the process and staying with too powerful keys.
  - grogenaut 4y ago
    Data plane volumes are absurd and then you have to crunch it. I estimated this for my company. But you really wouldn't want to pay to do all that cloud trail processing yourself. We have some buckets where the cloud trail storage with 1y retention is more than the bucket itself.
    - krab 4y ago
      Hmm, what I had in mind was rather a tool that would generate a new access key (or some other way to correlate the requests) and record and show me in real time what permissions I need and let me play with that and simulate what API calls would fail with a policy that I'm just writing. Very low volume.
      - grogenaut 4y ago
        I think that predicates you know everything your libraries, pulled in containers, kube charts, etc are doing... I'd love it if I could just be told what I'm doing and see 'yeah that makes sense' and be alerted to changes.
alex-olivier 4y ago
(Disclaimer: I work for Cerbos[1]) When we started working on Cerbos[1] the very first external bit of tooling we released was the Cerbos Playground[2] which does exactly what you say - allows you to see the requests and responses as they make changes to their policies, making it easier to test and refine their rules. This is a great starting point to prototype and test whether a decoupled authorization system is right for your use case. Cerbos uses YAML rather than a custom DSL to try and address the fact that authorization requirements generally don't sit with developers, rather a product owner of some sorts who is going to want to be able to comprehend the logic behind the permissions model. [1] https://cerbos.dev https://cerbos.dev [2] https://play.cerbos.dev https://play.cerbos.dev
ctvo 4y ago
Cedar has a playground that supports much of what you're asking for or were you referring to IAM (AWS specific, and not model policies for your business). https://www.cedarpolicy.com/en/playground https://www.cedarpolicy.com/en/playground
- ctvo 4y ago
  Interesting thing about the playground: the entire evaluator is apparently in a WebAssembly binary located at https://www.cedarpolicy.com/waterford_wasm_bg.wasm https://www.cedarpolicy.com/waterford_wasm_bg.wasm It gives me a little hope on the direction and ability to embed Cedar in non-AWS contexts.
- krab 4y ago
  It seems more akin to the IAM policy simulator to me. You need to know the specific action and context to play with the policy. In the real case I don't know what context the actions carry that I may use for filtering, I don't know the action names and I may not even know the full list of API calls the tool I use wants to make. AWS knows (could record) all of the above for me.
  - ctvo 4y ago
    > In the real case I don't know what context the actions carry that I may use for filtering You do know all of this with Cedar as the service owner though. You know the attributes on the entities, you know the policies. Something is going over my head, because I don't think I understand the use case. Can you give a concrete example?
    - krab 4y ago
      Probably we're looking at the problem from a different point of view. Imagine I'm a new AWS customer that creates their account, starts an Elastic Beanstalk application and tries to automate deployment via CI. The access key will need some permissions to EB, EC2, S3, maybe RDS, ECR... The best thing I can do at the moment is to expect that an example policy is somewhere in the docs.

by krab 4y ago

rollthehard6 4y ago

So much this, I don't understand why cloud providers don't offer an easy way to view this sort of thing as really it should all be in the logging anyway.

lozenge 4y ago
It seems like AWS isn't good with cross cutting concerns. Eg instead of Cloudformation being part of a service during development, that they could even use during testing the service, every team creates a custom api and cf is an afterthought. It wouldn't surprise me if every service is also evaluating against iam policies independently as well.
- rollthehard6 4y ago
  That reminds me of the latter stages of Sun Microsystems, where it seemed like multiple teams were developing competing solutions for system management and patching. Seems a wasteful model and not one that is servicing customers.
- grogenaut 4y ago
  Much of the services were built well before clod formation... Or cloud formation became popular... Or before everyone was willing to make the trade-off that features come later because they had to have cloud formation. Like any other best practice you invent along the way. I'll bet there's something about your 1/3/5/10/15 year old that isn't up to current snuff. People forget, AWS is massive and taking care of edge case after edge case while being in a bullet point war with other cloud providers.
  - erik_seaberg 4y ago
    Even new features sometimes don’t get CloudFormation support for quite a while (sometimes longer than third-party support like Terraform). I keep wondering why they didn’t want it for their own integration testing.
cperciva 4y ago
The execution of rulesets needs to be performed synchronously and as fast as possible, since requests are blocked until they pass the permission checks; but it doesn't need any sort of replication since each request is validated independently. Request logging can be performed asynchronously and has much more relaxed latency requirements, but data needs to be aggregated and replicated for durability. The extremely different requirements pretty much guarantee that these will be two completely separate systems; by Conway's Law, this means two completely separate teams. In practice, logging is more aimed at business analytics and billing. You're right that the information should be available, and there's no technical reason why it can't be made available, but there are technical reasons which influence the social reasons why it's not available.
fulafel 4y ago
AWS had the policy simulator for IAM policies but it hasn't been kept up to date wrt new features (like SCP)... Maybe they'll have something similar for this.

lockedinspace 4y ago

You can simulate a policy with AWS IAM Policy Simulator

krab 4y ago
This doesn't cut it. That tool helps if your role/user has a lot of policies that might interact between each other. But you already need to know the exact "actions" and context. On the other hand, I have a tool that calls some AWS services that may in turn call other AWS services. Now if something fails because of IAM denial, I have to go through the logs to figure out what it needed, sometimes it's not even in the logs (S3). Then add it to the policy and repeat to see the next failed call. I imagine being able to call real API calls, make them succeed and record what permissions it needed for each of those API calls. I don't need that as a permanent log, just as a development tool.
- sickmate 4y ago
  You can do this with Localstack, however it's gated behind their pro service. https://docs.localstack.cloud/user-guide/aws/iam/#explainable-iam https://docs.localstack.cloud/user-guide/aws/iam/#explainabl...
- kapilvt 4y ago
  All of the sdks support client side monitoring (CSM), so these sort of tools can be built client side. https://boto3.amazonaws.com/v1/documentation/api/1.10.46/guide/sdk-metrics.html#enable-sdk-metrics https://boto3.amazonaws.com/v1/documentation/api/1.10.46/gui... afaics the only challenge is mapping some of the apis to iam as its only 85% 1:1 There's also tools for helping with iam like (generator, and linter) https://github.com/salesforce/policy_sentry https://github.com/salesforce/policy_sentry https://github.com/duo-labs/parliament https://github.com/duo-labs/parliament
  - krab 4y ago
    > mapping some of the apis to iam as its only 85% 1:1 Yeah. Try to create an Elastic beanstalk app with only EB permissions.

ProjectMoonShot 4y ago

The book AWS Cookbook section 1.2 is on 'Generating a Least Privilege IAM Policy Based on Access Patterns'. [1] https://www.oreilly.com/library/view/aws-cookbook/9781492092599/ https://www.oreilly.com/library/view/aws-cookbook/9781492092... [2] https://docs.aws.amazon.com/IAM/latest/UserGuide/access-analyzer-policy-generation.html https://docs.aws.amazon.com/IAM/latest/UserGuide/access-anal...

krab 4y ago
This goes in the right direction. Still it's not 100% reliable (e.g. S3 isn't logged by default to cloudtrail, probably due to volume). Also, not really interactive. I'm talking mostly about ergonomics. Obviously we can solve the issue and create the right policy - but it's not as easy as it could be. That in turn leads to some people skipping the process and staying with too powerful keys.
- grogenaut 4y ago
  Data plane volumes are absurd and then you have to crunch it. I estimated this for my company. But you really wouldn't want to pay to do all that cloud trail processing yourself. We have some buckets where the cloud trail storage with 1y retention is more than the bucket itself.
  - krab 4y ago
    Hmm, what I had in mind was rather a tool that would generate a new access key (or some other way to correlate the requests) and record and show me in real time what permissions I need and let me play with that and simulate what API calls would fail with a policy that I'm just writing. Very low volume.
    - grogenaut 4y ago
      I think that predicates you know everything your libraries, pulled in containers, kube charts, etc are doing... I'd love it if I could just be told what I'm doing and see 'yeah that makes sense' and be alerted to changes.

alex-olivier 4y ago

(Disclaimer: I work for Cerbos[1]) When we started working on Cerbos[1] the very first external bit of tooling we released was the Cerbos Playground[2] which does exactly what you say - allows you to see the requests and responses as they make changes to their policies, making it easier to test and refine their rules. This is a great starting point to prototype and test whether a decoupled authorization system is right for your use case. Cerbos uses YAML rather than a custom DSL to try and address the fact that authorization requirements generally don't sit with developers, rather a product owner of some sorts who is going to want to be able to comprehend the logic behind the permissions model. [1] https://cerbos.dev https://cerbos.dev [2] https://play.cerbos.dev https://play.cerbos.dev

ctvo 4y ago

Cedar has a playground that supports much of what you're asking for or were you referring to IAM (AWS specific, and not model policies for your business). https://www.cedarpolicy.com/en/playground https://www.cedarpolicy.com/en/playground

ctvo 4y ago
Interesting thing about the playground: the entire evaluator is apparently in a WebAssembly binary located at https://www.cedarpolicy.com/waterford_wasm_bg.wasm https://www.cedarpolicy.com/waterford_wasm_bg.wasm It gives me a little hope on the direction and ability to embed Cedar in non-AWS contexts.
krab 4y ago
It seems more akin to the IAM policy simulator to me. You need to know the specific action and context to play with the policy. In the real case I don't know what context the actions carry that I may use for filtering, I don't know the action names and I may not even know the full list of API calls the tool I use wants to make. AWS knows (could record) all of the above for me.
- ctvo 4y ago
  > In the real case I don't know what context the actions carry that I may use for filtering You do know all of this with Cedar as the service owner though. You know the attributes on the entities, you know the policies. Something is going over my head, because I don't think I understand the use case. Can you give a concrete example?
  - krab 4y ago
    Probably we're looking at the problem from a different point of view. Imagine I'm a new AWS customer that creates their account, starts an Elastic Beanstalk application and tries to automate deployment via CI. The access key will need some permissions to EB, EC2, S3, maybe RDS, ECR... The best thing I can do at the moment is to expect that an example policy is somewhere in the docs.