4 ms·

Another reason why microservices are a good thing. They result in micro level permissions for individual resources. In saying that though, k8s in AWS was reall

Another reason why microservices are a good thing. They result in micro level permissions for individual resources. In saying that though, k8s in AWS was really shit at limiting what IAM roles containers could assume without it being the instance role. Crap like kube2iam and kiam came out to butcher the AWS metadata/instance networking. Thankfully AWS solved it with their new OIDC IDP. True DevOps culture workplaces have Devs writing their IAM roles as they know what services the app might talk to. Keen to try out some tools mentioned by the twitter person

lostmyoldone 6y ago
I don't see how microservices
lostmyoldone 6y ago
I don't see how microservices necessarily lead to what you call micro level permissions. If anything it's IAAS/PAAS and the inherent requirements that springs from that, to explicitly manage access that has started to drive this. Still, some deployment environments makes it extremely tedious to actually manage fine grained, least necessary privilege access. Especially for smaller outfits, setting up all the security specs for a quite typical setup at a good granularity has a lot of the feel of writing assembler code for a mcu with a bad datasheet. Figuring out which rights you actually need is sometimes hilariously convoluted, as examples often use excessively large scopes, and sometimes even figuring out which service to attach them to can be extremely non-obvious. I hope that in time there'll be tools on top of the k8s specs that takes these chores out of the equation, maybe there already are? I haven't tracked k8s closely, as it seems to mostly cater to larger outfits as of now.
- tedk-42 6y ago
  If your microservice only updates billing details on a dynamodb table, it'll never be vulnerable to having someone take over it and stealing all the data from it. It's what I call a micro level permission. The principal/actor can only write to one resource. Where places get it wrong is when their application writes to dynamodb, reads from S3, does a scan on another table etc. It leads Devs to making overly permissive permissions while they debug why their app isn't working like it use to.
meritt 6y ago
You can just as easily give overly permissive IAM credentials to a microservice. That's what happened here. Some tiny little web service was breached, and that service was provisioned a full-access IAM credential.
- tedk-42 6y ago
  If you're deploying a microservice on an individual ER2 inctance with an instance role, you're doing it wrong. Not sure where you got the idea it was a microservice. Names like a dodgy app server of some kind.
  - meritt 6y ago
    The point I am trying to make is "microservice" does nothing whatsoever to solve the problem of people incorrectly provisioning IAM permissions.
    - tedk-42 6y ago
      Nor does it in make developers better at their jobs. The point i'm trying to make is if the scope of what a system can do is limited, it's permission boundary/model is easy to define. Many things led to the incorrect provisioning of the IAM role. Lake of understanding of IAM for starters as well as consequences around it. By no means am I saying microservices would solve the problem. But it sure does make it easier to define what permissions your app needs to have as well as limit the blast radius of what is exposed when done correctly. This is impossible with monoliths on EC2 instances.
heavenlyblue 6y ago
Is that a pun? :)
mden 6y ago
Capital One uses microservices with multiple layers for privileging and is "true devops" with devs typically deciding IAM policies. One of the big issues with microservices is that you need to invest a whole lot more into infrastructure and tooling to be able to run them sanely. Most companies are really hesitant to invest sufficiently which leads to burn out, low productivity, and various ultimately avoidable bugs. This compounded with microservices being inherently more complex you often end up with tangled and messy products that are hard to understand software wise which also impacts security. Having worked at Capital One I think that's certainly one of the big issues at play. For devops, I personally don't get it, maybe I just haven't seen it done well or maybe when it's done well it just disappears into the background. But expecting all developers to be good at application coding and also have an in depth understanding of AWS security best practices is unreasonable. Something is wrong with the levels of abstractions being worked at.

lostmyoldone 6y ago

I don't see how microservices

lostmyoldone 6y ago

I don't see how microservices necessarily lead to what you call micro level permissions. If anything it's IAAS/PAAS and the inherent requirements that springs from that, to explicitly manage access that has started to drive this. Still, some deployment environments makes it extremely tedious to actually manage fine grained, least necessary privilege access. Especially for smaller outfits, setting up all the security specs for a quite typical setup at a good granularity has a lot of the feel of writing assembler code for a mcu with a bad datasheet. Figuring out which rights you actually need is sometimes hilariously convoluted, as examples often use excessively large scopes, and sometimes even figuring out which service to attach them to can be extremely non-obvious. I hope that in time there'll be tools on top of the k8s specs that takes these chores out of the equation, maybe there already are? I haven't tracked k8s closely, as it seems to mostly cater to larger outfits as of now.

tedk-42 6y ago
If your microservice only updates billing details on a dynamodb table, it'll never be vulnerable to having someone take over it and stealing all the data from it. It's what I call a micro level permission. The principal/actor can only write to one resource. Where places get it wrong is when their application writes to dynamodb, reads from S3, does a scan on another table etc. It leads Devs to making overly permissive permissions while they debug why their app isn't working like it use to.

meritt 6y ago

You can just as easily give overly permissive IAM credentials to a microservice. That's what happened here. Some tiny little web service was breached, and that service was provisioned a full-access IAM credential.

tedk-42 6y ago
If you're deploying a microservice on an individual ER2 inctance with an instance role, you're doing it wrong. Not sure where you got the idea it was a microservice. Names like a dodgy app server of some kind.
- meritt 6y ago
  The point I am trying to make is "microservice" does nothing whatsoever to solve the problem of people incorrectly provisioning IAM permissions.
  - tedk-42 6y ago
    Nor does it in make developers better at their jobs. The point i'm trying to make is if the scope of what a system can do is limited, it's permission boundary/model is easy to define. Many things led to the incorrect provisioning of the IAM role. Lake of understanding of IAM for starters as well as consequences around it. By no means am I saying microservices would solve the problem. But it sure does make it easier to define what permissions your app needs to have as well as limit the blast radius of what is exposed when done correctly. This is impossible with monoliths on EC2 instances.

heavenlyblue 6y ago

Is that a pun? :)

mden 6y ago

Capital One uses microservices with multiple layers for privileging and is "true devops" with devs typically deciding IAM policies. One of the big issues with microservices is that you need to invest a whole lot more into infrastructure and tooling to be able to run them sanely. Most companies are really hesitant to invest sufficiently which leads to burn out, low productivity, and various ultimately avoidable bugs. This compounded with microservices being inherently more complex you often end up with tangled and messy products that are hard to understand software wise which also impacts security. Having worked at Capital One I think that's certainly one of the big issues at play. For devops, I personally don't get it, maybe I just haven't seen it done well or maybe when it's done well it just disappears into the background. But expecting all developers to be good at application coding and also have an in depth understanding of AWS security best practices is unreasonable. Something is wrong with the levels of abstractions being worked at.