10 ms·

AWS IAM Roles, a tale of unnecessary complexity

255 points by wglb 4y ago

mkl95 4y ago
That logic is complex, but not that much. The industry is full of RBAC implementations that are orders of magnitude worse, and don't necessarily do what their owners think they do.
brycelarkin 4y ago
The CDK has made managing IAM so much easier for applications. It’s one of the main reasons we moved from Terraform to CDK.
- theideaofcoffee 4y ago
  We did the opposite because there was so much obfuscation about what exactly CDK was doing behind the curtains with respect to "small" things like IAM. We needed to know exactly which role was created or modified, etc, and we just couldn't get that with the basic interfaces that CDK provided. Writing those roles, users, groups, policies, attachments out explicitly into their own resource statements made things so much more clear, especially with respect to the relationships to other resources, and less risky
  - bsjaux628 4y ago
    You can be pretty granular in CDK, just skip the L3 constructs and build everything using the L2 one or if you hate yourself go for the L1 which is manipulating the Cloudformarion template withing CDK
  - digianarchist 4y ago
    Why not both? https://github.com/hashicorp/terraform-cdk https://github.com/hashicorp/terraform-cdk
  - indrora 4y ago
    You can, but it takes a little bit of wringing your hands. You can inspect the entire construct tree that is created (minus anything created by Custom Resources, which are opaque to the CDK) via the Aspects visitor pattern. It allows you to audit the entire tree of resources. The Annotations framework allows you to add flags (notes, warnings, and errors) to any object in the construct hierarchy.
- dangus 4y ago
  How does CDK compare to Pulumi? It looks like a similar concept. I haven’t tried out either.
  - alwaysanon 4y ago
    CDK 'compiles' CloudFormation templates basically making it much easier to write using TypeScript/Python/Java/C# instead of JSON/YAML. The real thing is does though is give you higher-level object-oriented constructs with best practices baked in. It has much more sensible defaults baked in and, almost ironically, the fewer parameters you pass to these classes the more opinionated CloudFormation comes out. The example that blew my mind is if you don't specify a password for RDS it provisions an AWS SecretsManager Secret, generates a random password and puts it in there and then tells the RDS to use that Secret. If you do specify a password it doesn't do that stuff. Lots of stuff like that - it turns encryption on by default and creates the keys if you don't specify, it creates private subnets and a NAT gateway for VPCs if you don't specify. It was basically "its too hard to fix the service APIs or their CloudFormation so we'll fix the problem outside of / on top of them with a tool users run on their laptops or in their pipelines to deal with generating the thousands of lines of CF boilerplate that are required to really do the right thing these days. Of course you can be very explicit in most of these constructs and the more explicit you are about what you want the less of its opinions happen.
    - abraxas 4y ago
      Unfortunately, a large problem with this lipstick on a pig approach that Amazon took with CDK is that the moment something fails to work you are right back to combing through Cloud Formation scripts. The abstraction is so leaky that I don't recall a single CDK project that I worked with where I didn't have to inspect the CF output and painstakingly map back to the CDK sources.
    - dangus 4y ago
      Thanks for that info. Apparently, I'm falling behind enough that I didn't even realize that Terraform had its own CDK as well. I personally agree with you about CloudFormation. I dislike everything about it. I think the people who build the Terraform and Pulumi ecosystems are plain and simple better architects than AWS employees. I make that relatively blind assumption because I've done an interview for AWS. They optimize their hiring process for hiring blindly loyal drones. They want you to memorize company values and relate everything you've ever done to each and every value. They make you read them off a list one-by-one. It felt so dystopian to me. AWS seems like the last place a creative, talented developer would want to go. The phone screen would scare most of them off.
alwaysanon 4y ago
The author was doing the right thing with explaining it only makes sense when you know the history of how we got to where we are - but there is a bit more to it. It is all a series of band-aids. Band-aids all the way down. As somebody who has been involved with AWS since 2012 I've seen them all get added incrementally in response to explosions in usage and complexity and customer unhappiness and frustration. Explicit allows being all you can do in an IAM policy were easy(ish) when there was a handful of AWS services and API actions. As there were more and more services and policy actions etc. they became unwieldy. Enter Permission Boundaries where you could wrap a few explicit denies around them. Kubernetes RBAC is nearly at the same place and could now really use those too - but I digress. Also, early on in my AWS journey, even two accounts (one non-prod and one prod) was only done half the time and viewed as a best practice to think about - people genrally just opened one AWS account. But when IAM wasn't enough (i.e. there wasn't enough granularity on the resources or the conditions exposed etc.) the answer became separate AWS accounts as the only, or at least the easiest, way enforce these authorization boundaries/separations with a blunt instrument you could trust. It also helped to keep your bills straight before they would do things like break them down by Tag. Then you often needed cross-account role assumptions to deal with the inevitable cases where things or people needed access between these accounts. Then the explosion in AWS Accounts led to AWS Organizations to provision and manage them all. And it built in Service Control Policies and OUs as a tool/layer to help further manage/constrain IAM policies/permissions (IAM policies, Permission Boundaries and SCPs now being in a venn diagram with each other these days). But AWS Organizations managing heaps of accounts was also too painful to use and get right and so they brought in AWS Control Tower to help make setting up Organizations easier. So, in short, this all makes sense when you understand the inability to totally rewrite/refactor important complex systems used by customers (breaking backward compatibility) and instead trying to keep solving all the challenges with an steady stream of incremental band-aids that you can announce at re:Invent.
- herdcall 4y ago
  What in your opinion is a good/best implementation then? We're currently engaged in re-designing our authorization flow, and I was planning to use a model quite similar to AWS IAM (policies, etc.). Contrary to this article, I actually thought the IAM model was simple in that we can tie each entity's access down to a set of policies that we can independently develop, store, and process for providing access.
sylens 4y ago
The author does a good job showing how AWS got to this point, but I think the majority of their argument is that they don't care for the semantics. Lines like: >>A consequence of this change was to further remove any coherent meaning for the term “role” and >>So what is an IAM role then? Simply put, it’s a principal (i.e. an identity) with no long-lived credentials, that can be impersonated for arbitrary purposes. Show that this is the main grievance. And to that I say - I hope you never try to mix Azure workloads at the same time as you are working on other cloud providers, as the nomenclature differences will drive you crazy. I do agree that the GCP model is more sensible (unless you have to 'unlearn' the AWS style first) but my major grievance is the combination and intersection of lots of different permission layers in AWS - you can have SCP's acting as a hard explicit deny at the top level, then your policies attached to roles/users, and then some services (such as S3) can have resource based policies as well. Even the best engineers can forget a setting or two and have to spend time troubleshooting, especially across a lot of accounts.
coderintherye 4y ago
Rant: I'm glad the author called out GCP as well, but the idea that it's the user's fault for granting too many permissions is absurd. Has the author tried doing this at scale? Try a super simple example: I want a user to have access to query a schema in BigQuery. Can you guess the magic combo of permissions that is going to take? Because the documentation is not going to correctly tell you. Thus after spending way too much time trying to get it done one ends up widely over-permissioning to just get the damn thing to work. You could argue their model is sound and it's just that how they aggregated permissions into roles is absurd or even just argue that they need better defined roles and I would accept that, but still it's quite frustrating as is. I mean just look at the length of the reference: https://cloud.google.com/iam/docs/understanding-roles https://cloud.google.com/iam/docs/understanding-roles
- systemvoltage 4y ago
  Yea I am not a fan of the Cloud and many of its complexities but if I have to be honest, when you have 5k employees working on AWS resources: yes, this complexity is actually requested by customers. It was built to meet their granular needs. Same thing with policies. To the author of the article: AWS is too complex? Then it is not for you. That said, various cloud providers have different levels of complexity and it is definitely up for debate and criticism.
  - SkyPuncher 4y ago
    The problem is all of these providers should have simple roll ups.
    - insanitybit 4y ago
      Roll ups have their own problems. In particular, you really don't want them changing over time. I have banned all of the "managed policies" at my company because AWS is free to add new permissions and you have no way of knowing.
      - SkyPuncher 4y ago
oneplane 4y ago
If you make the choice to say account == project, you're good to go in most cases. If you need deep permission hierarchies, you're screwed unless you go ABAC, but that is not amazingly implemented in any of the big clouds. Another saving scenario might be using a public cloud but not using it for flexible deployments and workloads (which are not the same thing and also indicates that a cloud may not be the best place to do this). Azure is the worst of all worlds because its Azure AD implementation is much better than classic AD but much worse than modern IAM. It's really weird how they made something that on the surface seems like a nice modern replacement but then ends up biting you in the ass as soon as you try to stop playing datacenter-in-the-cloud and actually do something new/innovative. Same goes for GCP and Azure in terms of service credentials when compared to IRSA and Instance Roles, or the other way around, non-standard credentials in AWS vs. OIDC or x509 in GCP (and Azure, and Kubernetes...). Granted, those aren't amazing either, but just for initial identities (after which you should switch into a specific principal - bare user principal policies deserve to be burned). The odd thing about most of this (in GCP and AWS, Azure seems to still have a two-faced API in the 'standard for me but not for thee' style) is that at some point resource interfaces got more and more standardised within each big cloud, yet the IAM part did not grow the same way. Perhaps because it wasn't possible to refactor/discard legacy systems in time (Just like EC2 Classic is only now going into some sort of read-only mode), or the scaling issues weren't apparent early enough? Some scaling solutions are a whole different can of worms, like the folder/inheritance structure which becomes the same mess as AWS with the boundary structure. That is one thing where a more classical "Directory" approach doesn't have this issue as much, but you trade it for the single-OU-leaf-at-a-time issue. Seeing how we do sometimes have to apply the ugliest of kludges for complex IAM scenarios (i.e. a microservice accessing two different resources in different scopes [ AWS accounts ] and some resources support resource-level policies to allow this but others are always IAM-local, so now you have to make your IaC pick the target that has the least flexible IAM context, create the principal there, and refer to that principal from all the remote contexts), you'd think instead of developing shrink-wrapping extensions more focus would be applied to ABAC and resource-local policies... Regardless of the implementation, I do question systems that cannot separate credentials from principals. A principal is what you are granted to BE post-authentication, and you need credentials for that. A policy grants a principal one or more ACTIONS. Therefore, a "user" should be credentials, but not a principal. But this is getting too much into semantics... as long as your authenticity is not tied to your authorisation the system should be better off (even if it ends up calling principals roles - just call users not-principals in that case, and disallow policy attachments or user references). A thing that does make more sense than the author of the article states is making things like this 'just another service'. When you don't do that, but make it 'special' instead, you create a single-root problem. An example is the problem where billing is either in the local account, or in the org account, and nowhere else (in AWS). In GCP your projects and billing accounts are just two resources that can exist in arbitrary numbers and be linked as needed. Billing shouldn't be special, just like IAM shouldn't be special.
imwillofficial 4y ago
I learned so much in this thread.
pram 4y ago
Ignoring all the warts of how schizophrenic the different AWS layers are, I personally like the policy document format. I wouldn’t mind a similar style at the OS/kernel level for control groups and other permissions of that type. Having a list of principals and allowed operations would be easier to reason about (imho) than systemd unit files, or selinux, or filesystem permissions, or firewall rules, or the rest of the mess which is Linux security. I think Fuchsia is doing something exactly like that?
- jerjerjer 4y ago
  To be fair selinux is a bar so low you'd have to dig to do worse.
- hedora 4y ago
  Unix groups are much, much easier to correctly configure than everything you mentioned, and at least an order of magnitude simpler as well.
Eleison23 4y ago
MULTICS reinvents itself. It's penguins all the way down.
deegles 4y ago
What are the best resources for getting better at working with IAM? Already pretty experienced but I feel like my mental model could be better.
scubbo 4y ago
I've worked on plenty of multi-account projects, and never felt a need to use Organizations. So far as I can tell from this, it allows you to attach permissions to roles in multiple sub-accounts - why wouldn't you be able to use CDK for that?
- Aeolun 4y ago
  You could, but Organisations predates heavy adoption of CDK. I imagine it also guarantees that the roles keep having the correct permissions.
- whoknew1122 4y ago
  Organizations also consolidates billing, which means you can (potentially) get usage discounts. There's a lot of reasons to use Organizations, but I really don't want to write an essay. But it allows things such as segregated security accounts, centralized logging, etc. It can help your security posture a bunch.
  - scubbo 4y ago
    Interesting! Technically I guess a lot of that is possible with raw CDK (centralized logging would be, for instance, if you created appropriate roles for each of the managed accounts), but it never hurts to have it done for you automatically! Sounds like it provides sensible default templates for things you _could_ roll by hand if you were feeling masochistic :) Plus, as the sibling comment said, it predated CDK so was probably the only option at the time.
oxplot 4y ago
Implementation of EVERY service on AWS is unnecessarily complicated. Not an exaggeration. Making things simple takes considerable consideration and effort, and that costs money. AWS is a nano-margin business. As long as something works and doesn't fall apart, they ship it.
TomK32 4y ago
A few years ago I was tasked with migrating a database from one AWS org to another and thanks to IAM we were able to allow this without me accessing anything else in the other org and migrate the database without any more support from the other org. Some time later I found a key in the database I migrated which turned out to give me full access to their AWS... So, be careful with it and check or even remove old keys after a few years.
- whoknew1122 4y ago
  Interestingly enough, this is one of the very use case for IAM roles. IAM roles' credentials have a maximum life of 12 hours. The article is complaining about the thing that exists to avoid the issue you point out.
  - deleted 4y ago
    [deleted]
Ambolia 4y ago
Almost ALL manuals for setting up something on AWS, third party tools, and sometimes AWS official tools, effectively have a step that requires admin permissions, like being able to attach arbitrary policies to an instance. At that point you can either: - give everybody Admin access - involve the same 2 or 3 trusted people in processes that they shouldn't really be involved - dig down into permissions to try to build something yourself that apparently the people writting the manuals gave up on doing Looks very bleak, anybody has a different view in this? Seems unlikely that big organizations work like this on AWS.
- gw98 4y ago
  No this is exactly how it works. There’s technically ideal policy and a bunch of exceptions which are behind a “break glass in case of emergency” account. Realistically nothing works however many hours you spend in IAM unless you use the emergency account so that’s what everyone does.
  - ownagefool 4y ago
    Right. In practice, you manually bootstrap your CI/CD, and if you really want to be secret squirrel secure, you can audit that. Everything going forward gets to be commited via code, peer review, least privledged access, and is automatically deployed by the robot. At that point, you might hand out some break glass credentials for a few items, or when your CI/CD goes down, but usage of that should alert and be manually reviewed. You now have a pretty decent setup, as everything is checked.
    - gw98 4y ago
      That's what we say we do. Reality is that most of the time the entire team is logged in via delegated admin accounts working out what the automation fucked up or deleting resources which don't want to go away when the automation craps out or futzing with things that are broken or missing in Terraform. Fully driving a cloud via code is a complete lie.
      - 4y ago
rossmohax 4y ago
Author keeps saying that Roles are identities,but they are exactly opposite. Role is a set of permissions token issued for that role will have. Whether identity can issue token for that role or not is governed separately. Role is not identity , nor service accounts nor principal.
- isbvhodnvemrwvn 4y ago
  In AWS parlance it is an identity: https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html
iamflimflam1 4y ago
Anyone complaining about AWS has obviously never had GCP inflicted on them…
- dima_vm 4y ago
  GCP is an order of magnitude better at IAM. And the article is actually exactly about that, with comparison.
  - mdaniel 4y ago
    I would like to disagree with that; GCP seems to do a much better job at permission granularity, but does a "punch me in the face" level worse job at IAM role binding to principals With AWS, if I have "role-A", used by "user1", and I now wish to bind it to "user2", do you know how many times I have to make reference to "user1"? ZERO With GCP, binding is authoritative, so if I don't copy-paste the "members" from the previous incantation of that roleBinding over, buh-bye to "user1". Stark-raving insanity
tarasglek 4y ago
The problem is that no one hierarchy fits every org. Sideways relationships that cross hierarchies exist, other exceptions exist. I find directed-graph-based approach in spicedb to be a better primitive than hierarchy with weird exceptions. Directed edges are a much easier way to think about relationships even if they are logically equivalent to hierarchies. https://github.com/authzed/spicedb https://github.com/authzed/spicedb
willtemperley 4y ago
From today's inbox seems like they're taking heed. I expect the following makes sense to someone: "We have updated our back-end permissions policy to simplify on-boarding to self-managed AWS CloudFormation StackSets. Starting October 31, 2022, AWS CloudFormation StackSets no longer requires sns* permissions in AWSCloudFormationStackSetExecutionRole as a prerequisite for getting started with self-managed StackSets. StackSets requires you to create a service role named AWSCloudFormationStackSetExecutionRole that trusts the customized administration role for each target account. Previously, you needed to provide sns:, s3:, and cloudformation:* permissions to AWSCloudFormationStackSetExecutionRole as prerequisites to allow StackSets to manage and provision resources on your behalf. Now, StackSets has removed the explicit need for sns:* permissions as prerequisites in AWSCloudFormationStackSetExecutionRole. Please note, you will still need to provide s3:* and cloudformation:* permissions in AWSCloudFormationStackSetExecutionRole."

255 points by wglb 4y ago

That logic is complex, but not that much. The industry is full of RBAC implementations that are orders of magnitude worse, and don't necessarily do what their owners think they do.

brycelarkin 4y ago

The CDK has made managing IAM so much easier for applications. It’s one of the main reasons we moved from Terraform to CDK.

theideaofcoffee 4y ago
We did the opposite because there was so much obfuscation about what exactly CDK was doing behind the curtains with respect to "small" things like IAM. We needed to know exactly which role was created or modified, etc, and we just couldn't get that with the basic interfaces that CDK provided. Writing those roles, users, groups, policies, attachments out explicitly into their own resource statements made things so much more clear, especially with respect to the relationships to other resources, and less risky
- bsjaux628 4y ago
  You can be pretty granular in CDK, just skip the L3 constructs and build everything using the L2 one or if you hate yourself go for the L1 which is manipulating the Cloudformarion template withing CDK
- digianarchist 4y ago
  Why not both? https://github.com/hashicorp/terraform-cdk https://github.com/hashicorp/terraform-cdk
- indrora 4y ago
  You can, but it takes a little bit of wringing your hands. You can inspect the entire construct tree that is created (minus anything created by Custom Resources, which are opaque to the CDK) via the Aspects visitor pattern. It allows you to audit the entire tree of resources. The Annotations framework allows you to add flags (notes, warnings, and errors) to any object in the construct hierarchy.
dangus 4y ago
How does CDK compare to Pulumi? It looks like a similar concept. I haven’t tried out either.
- alwaysanon 4y ago
  CDK 'compiles' CloudFormation templates basically making it much easier to write using TypeScript/Python/Java/C# instead of JSON/YAML. The real thing is does though is give you higher-level object-oriented constructs with best practices baked in. It has much more sensible defaults baked in and, almost ironically, the fewer parameters you pass to these classes the more opinionated CloudFormation comes out. The example that blew my mind is if you don't specify a password for RDS it provisions an AWS SecretsManager Secret, generates a random password and puts it in there and then tells the RDS to use that Secret. If you do specify a password it doesn't do that stuff. Lots of stuff like that - it turns encryption on by default and creates the keys if you don't specify, it creates private subnets and a NAT gateway for VPCs if you don't specify. It was basically "its too hard to fix the service APIs or their CloudFormation so we'll fix the problem outside of / on top of them with a tool users run on their laptops or in their pipelines to deal with generating the thousands of lines of CF boilerplate that are required to really do the right thing these days. Of course you can be very explicit in most of these constructs and the more explicit you are about what you want the less of its opinions happen.
  - abraxas 4y ago
    Unfortunately, a large problem with this lipstick on a pig approach that Amazon took with CDK is that the moment something fails to work you are right back to combing through Cloud Formation scripts. The abstraction is so leaky that I don't recall a single CDK project that I worked with where I didn't have to inspect the CF output and painstakingly map back to the CDK sources.
  - dangus 4y ago
    Thanks for that info. Apparently, I'm falling behind enough that I didn't even realize that Terraform had its own CDK as well. I personally agree with you about CloudFormation. I dislike everything about it. I think the people who build the Terraform and Pulumi ecosystems are plain and simple better architects than AWS employees. I make that relatively blind assumption because I've done an interview for AWS. They optimize their hiring process for hiring blindly loyal drones. They want you to memorize company values and relate everything you've ever done to each and every value. They make you read them off a list one-by-one. It felt so dystopian to me. AWS seems like the last place a creative, talented developer would want to go. The phone screen would scare most of them off.

alwaysanon 4y ago

The author was doing the right thing with explaining it only makes sense when you know the history of how we got to where we are - but there is a bit more to it. It is all a series of band-aids. Band-aids all the way down. As somebody who has been involved with AWS since 2012 I've seen them all get added incrementally in response to explosions in usage and complexity and customer unhappiness and frustration. Explicit allows being all you can do in an IAM policy were easy(ish) when there was a handful of AWS services and API actions. As there were more and more services and policy actions etc. they became unwieldy. Enter Permission Boundaries where you could wrap a few explicit denies around them. Kubernetes RBAC is nearly at the same place and could now really use those too - but I digress. Also, early on in my AWS journey, even two accounts (one non-prod and one prod) was only done half the time and viewed as a best practice to think about - people genrally just opened one AWS account. But when IAM wasn't enough (i.e. there wasn't enough granularity on the resources or the conditions exposed etc.) the answer became separate AWS accounts as the only, or at least the easiest, way enforce these authorization boundaries/separations with a blunt instrument you could trust. It also helped to keep your bills straight before they would do things like break them down by Tag. Then you often needed cross-account role assumptions to deal with the inevitable cases where things or people needed access between these accounts. Then the explosion in AWS Accounts led to AWS Organizations to provision and manage them all. And it built in Service Control Policies and OUs as a tool/layer to help further manage/constrain IAM policies/permissions (IAM policies, Permission Boundaries and SCPs now being in a venn diagram with each other these days). But AWS Organizations managing heaps of accounts was also too painful to use and get right and so they brought in AWS Control Tower to help make setting up Organizations easier. So, in short, this all makes sense when you understand the inability to totally rewrite/refactor important complex systems used by customers (breaking backward compatibility) and instead trying to keep solving all the challenges with an steady stream of incremental band-aids that you can announce at re:Invent.

herdcall 4y ago
What in your opinion is a good/best implementation then? We're currently engaged in re-designing our authorization flow, and I was planning to use a model quite similar to AWS IAM (policies, etc.). Contrary to this article, I actually thought the IAM model was simple in that we can tie each entity's access down to a set of policies that we can independently develop, store, and process for providing access.

sylens 4y ago

The author does a good job showing how AWS got to this point, but I think the majority of their argument is that they don't care for the semantics. Lines like: >>A consequence of this change was to further remove any coherent meaning for the term “role” and >>So what is an IAM role then? Simply put, it’s a principal (i.e. an identity) with no long-lived credentials, that can be impersonated for arbitrary purposes. Show that this is the main grievance. And to that I say - I hope you never try to mix Azure workloads at the same time as you are working on other cloud providers, as the nomenclature differences will drive you crazy. I do agree that the GCP model is more sensible (unless you have to 'unlearn' the AWS style first) but my major grievance is the combination and intersection of lots of different permission layers in AWS - you can have SCP's acting as a hard explicit deny at the top level, then your policies attached to roles/users, and then some services (such as S3) can have resource based policies as well. Even the best engineers can forget a setting or two and have to spend time troubleshooting, especially across a lot of accounts.

coderintherye 4y ago

Rant: I'm glad the author called out GCP as well, but the idea that it's the user's fault for granting too many permissions is absurd. Has the author tried doing this at scale? Try a super simple example: I want a user to have access to query a schema in BigQuery. Can you guess the magic combo of permissions that is going to take? Because the documentation is not going to correctly tell you. Thus after spending way too much time trying to get it done one ends up widely over-permissioning to just get the damn thing to work. You could argue their model is sound and it's just that how they aggregated permissions into roles is absurd or even just argue that they need better defined roles and I would accept that, but still it's quite frustrating as is. I mean just look at the length of the reference: https://cloud.google.com/iam/docs/understanding-roles https://cloud.google.com/iam/docs/understanding-roles

systemvoltage 4y ago
Yea I am not a fan of the Cloud and many of its complexities but if I have to be honest, when you have 5k employees working on AWS resources: yes, this complexity is actually requested by customers. It was built to meet their granular needs. Same thing with policies. To the author of the article: AWS is too complex? Then it is not for you. That said, various cloud providers have different levels of complexity and it is definitely up for debate and criticism.
- SkyPuncher 4y ago
  The problem is all of these providers should have simple roll ups.
  - insanitybit 4y ago
    Roll ups have their own problems. In particular, you really don't want them changing over time. I have banned all of the "managed policies" at my company because AWS is free to add new permissions and you have no way of knowing.
    - SkyPuncher 4y ago

oneplane 4y ago

If you make the choice to say account == project, you're good to go in most cases. If you need deep permission hierarchies, you're screwed unless you go ABAC, but that is not amazingly implemented in any of the big clouds. Another saving scenario might be using a public cloud but not using it for flexible deployments and workloads (which are not the same thing and also indicates that a cloud may not be the best place to do this). Azure is the worst of all worlds because its Azure AD implementation is much better than classic AD but much worse than modern IAM. It's really weird how they made something that on the surface seems like a nice modern replacement but then ends up biting you in the ass as soon as you try to stop playing datacenter-in-the-cloud and actually do something new/innovative. Same goes for GCP and Azure in terms of service credentials when compared to IRSA and Instance Roles, or the other way around, non-standard credentials in AWS vs. OIDC or x509 in GCP (and Azure, and Kubernetes...). Granted, those aren't amazing either, but just for initial identities (after which you should switch into a specific principal - bare user principal policies deserve to be burned). The odd thing about most of this (in GCP and AWS, Azure seems to still have a two-faced API in the 'standard for me but not for thee' style) is that at some point resource interfaces got more and more standardised within each big cloud, yet the IAM part did not grow the same way. Perhaps because it wasn't possible to refactor/discard legacy systems in time (Just like EC2 Classic is only now going into some sort of read-only mode), or the scaling issues weren't apparent early enough? Some scaling solutions are a whole different can of worms, like the folder/inheritance structure which becomes the same mess as AWS with the boundary structure. That is one thing where a more classical "Directory" approach doesn't have this issue as much, but you trade it for the single-OU-leaf-at-a-time issue. Seeing how we do sometimes have to apply the ugliest of kludges for complex IAM scenarios (i.e. a microservice accessing two different resources in different scopes [ AWS accounts ] and some resources support resource-level policies to allow this but others are always IAM-local, so now you have to make your IaC pick the target that has the least flexible IAM context, create the principal there, and refer to that principal from all the remote contexts), you'd think instead of developing shrink-wrapping extensions more focus would be applied to ABAC and resource-local policies... Regardless of the implementation, I do question systems that cannot separate credentials from principals. A principal is what you are granted to BE post-authentication, and you need credentials for that. A policy grants a principal one or more ACTIONS. Therefore, a "user" should be credentials, but not a principal. But this is getting too much into semantics... as long as your authenticity is not tied to your authorisation the system should be better off (even if it ends up calling principals roles - just call users not-principals in that case, and disallow policy attachments or user references). A thing that does make more sense than the author of the article states is making things like this 'just another service'. When you don't do that, but make it 'special' instead, you create a single-root problem. An example is the problem where billing is either in the local account, or in the org account, and nowhere else (in AWS). In GCP your projects and billing accounts are just two resources that can exist in arbitrary numbers and be linked as needed. Billing shouldn't be special, just like IAM shouldn't be special.

imwillofficial 4y ago

I learned so much in this thread.

pram 4y ago

Ignoring all the warts of how schizophrenic the different AWS layers are, I personally like the policy document format. I wouldn’t mind a similar style at the OS/kernel level for control groups and other permissions of that type. Having a list of principals and allowed operations would be easier to reason about (imho) than systemd unit files, or selinux, or filesystem permissions, or firewall rules, or the rest of the mess which is Linux security. I think Fuchsia is doing something exactly like that?

jerjerjer 4y ago
To be fair selinux is a bar so low you'd have to dig to do worse.
hedora 4y ago
Unix groups are much, much easier to correctly configure than everything you mentioned, and at least an order of magnitude simpler as well.

Eleison23 4y ago

MULTICS reinvents itself. It's penguins all the way down.

deegles 4y ago

What are the best resources for getting better at working with IAM? Already pretty experienced but I feel like my mental model could be better.

scubbo 4y ago

I've worked on plenty of multi-account projects, and never felt a need to use Organizations. So far as I can tell from this, it allows you to attach permissions to roles in multiple sub-accounts - why wouldn't you be able to use CDK for that?

Aeolun 4y ago
You could, but Organisations predates heavy adoption of CDK. I imagine it also guarantees that the roles keep having the correct permissions.
whoknew1122 4y ago
Organizations also consolidates billing, which means you can (potentially) get usage discounts. There's a lot of reasons to use Organizations, but I really don't want to write an essay. But it allows things such as segregated security accounts, centralized logging, etc. It can help your security posture a bunch.
- scubbo 4y ago
  Interesting! Technically I guess a lot of that is possible with raw CDK (centralized logging would be, for instance, if you created appropriate roles for each of the managed accounts), but it never hurts to have it done for you automatically! Sounds like it provides sensible default templates for things you _could_ roll by hand if you were feeling masochistic :) Plus, as the sibling comment said, it predated CDK so was probably the only option at the time.

oxplot 4y ago

Implementation of EVERY service on AWS is unnecessarily complicated. Not an exaggeration. Making things simple takes considerable consideration and effort, and that costs money. AWS is a nano-margin business. As long as something works and doesn't fall apart, they ship it.

TomK32 4y ago

A few years ago I was tasked with migrating a database from one AWS org to another and thanks to IAM we were able to allow this without me accessing anything else in the other org and migrate the database without any more support from the other org. Some time later I found a key in the database I migrated which turned out to give me full access to their AWS... So, be careful with it and check or even remove old keys after a few years.

whoknew1122 4y ago
Interestingly enough, this is one of the very use case for IAM roles. IAM roles' credentials have a maximum life of 12 hours. The article is complaining about the thing that exists to avoid the issue you point out.
- deleted 4y ago
  [deleted]

Ambolia 4y ago

Almost ALL manuals for setting up something on AWS, third party tools, and sometimes AWS official tools, effectively have a step that requires admin permissions, like being able to attach arbitrary policies to an instance. At that point you can either: - give everybody Admin access - involve the same 2 or 3 trusted people in processes that they shouldn't really be involved - dig down into permissions to try to build something yourself that apparently the people writting the manuals gave up on doing Looks very bleak, anybody has a different view in this? Seems unlikely that big organizations work like this on AWS.

gw98 4y ago
No this is exactly how it works. There’s technically ideal policy and a bunch of exceptions which are behind a “break glass in case of emergency” account. Realistically nothing works however many hours you spend in IAM unless you use the emergency account so that’s what everyone does.
- ownagefool 4y ago
  Right. In practice, you manually bootstrap your CI/CD, and if you really want to be secret squirrel secure, you can audit that. Everything going forward gets to be commited via code, peer review, least privledged access, and is automatically deployed by the robot. At that point, you might hand out some break glass credentials for a few items, or when your CI/CD goes down, but usage of that should alert and be manually reviewed. You now have a pretty decent setup, as everything is checked.
  - gw98 4y ago
    That's what we say we do. Reality is that most of the time the entire team is logged in via delegated admin accounts working out what the automation fucked up or deleting resources which don't want to go away when the automation craps out or futzing with things that are broken or missing in Terraform. Fully driving a cloud via code is a complete lie.
    - 4y ago

rossmohax 4y ago

Author keeps saying that Roles are identities,but they are exactly opposite. Role is a set of permissions token issued for that role will have. Whether identity can issue token for that role or not is governed separately. Role is not identity , nor service accounts nor principal.

isbvhodnvemrwvn 4y ago
In AWS parlance it is an identity: https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html

iamflimflam1 4y ago

Anyone complaining about AWS has obviously never had GCP inflicted on them…

dima_vm 4y ago
GCP is an order of magnitude better at IAM. And the article is actually exactly about that, with comparison.
- mdaniel 4y ago
  I would like to disagree with that; GCP seems to do a much better job at permission granularity, but does a "punch me in the face" level worse job at IAM role binding to principals With AWS, if I have "role-A", used by "user1", and I now wish to bind it to "user2", do you know how many times I have to make reference to "user1"? ZERO With GCP, binding is authoritative, so if I don't copy-paste the "members" from the previous incantation of that roleBinding over, buh-bye to "user1". Stark-raving insanity

tarasglek 4y ago

The problem is that no one hierarchy fits every org. Sideways relationships that cross hierarchies exist, other exceptions exist. I find directed-graph-based approach in spicedb to be a better primitive than hierarchy with weird exceptions. Directed edges are a much easier way to think about relationships even if they are logically equivalent to hierarchies. https://github.com/authzed/spicedb https://github.com/authzed/spicedb

willtemperley 4y ago

From today's inbox seems like they're taking heed. I expect the following makes sense to someone: "We have updated our back-end permissions policy to simplify on-boarding to self-managed AWS CloudFormation StackSets. Starting October 31, 2022, AWS CloudFormation StackSets no longer requires sns* permissions in AWSCloudFormationStackSetExecutionRole as a prerequisite for getting started with self-managed StackSets. StackSets requires you to create a service role named AWSCloudFormationStackSetExecutionRole that trusts the customized administration role for each target account. Previously, you needed to provide sns:, s3:, and cloudformation:* permissions to AWSCloudFormationStackSetExecutionRole as prerequisites to allow StackSets to manage and provision resources on your behalf. Now, StackSets has removed the explicit need for sns:* permissions as prerequisites in AWSCloudFormationStackSetExecutionRole. Please note, you will still need to provide s3:* and cloudformation:* permissions in AWSCloudFormationStackSetExecutionRole."