4 ms·

Do you think humans are doing a better job? Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason

Do you think humans are doing a better job? Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools. Why not use Slauth and other checks such as policy simulators to get tightened policies pre-deployed

jsploit 3y ago
> Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools. It'd potentially cost millions more to recover from a GPT-4 disaster.
verdverm 3y ago
I'm not your target user, I don't feel the priority on this problem even though our permissions are more permissive than we'd like. Thing is, to rein them in typically requires application changes. You cannot just sprinkle magic LLM dust on IAM and make things better. My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM and system architecture to know if what the LLM is saying is correct, so where does that leave us? I think they can be an extra pair of eyes, but not the driver. Still, there is a signal to noise problem that remains, due to the inherent hallucinations.
- thehucklecat 3y ago
  what kind of application changes are you thinking it would equire? my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).
  - verdverm 3y ago
    Some scenarios 1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account. 2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production 3. You have to change human behaviors and habits (this is the real hard one). I've had to revert changes because the increased security blocked developers and they don't have time to adapt for the next deadline. 4. Getting parity in local development workflows is also challenging. How and where do you match vs except from IAM parity? 5. Should I give the current server access to a particular cloud service/resource or break out that particular function into a lambda and minimize the permissions there? You have to think through the implications of a breach and how/where you want to limit the blast radius. 6. This is probably obvious, but implementing application level controls, like API endpoint permissioning. IAM is not limited to cloud infra
    - DanielSlauth 3y ago
      The open-source project is a CLI you can put into your CI/CD so i think a pretty neat workflow where there should be less friction considering DevOps/security don't need to ping-pong on permissions.
      - verdverm 3y ago
        when you keep telling, you ain't selling ask questions to deepen your understanding > ...ping-pong... It was a scheduling problem rather than a decision problem. The impact radius is always more than you anticipate
- DanielSlauth 3y ago
  First of all its pretty awesome your permissions are very tight. You are definitely on the other side of the spectrum compared to the rest. I get it that there is a lot of skepticism because of people hyping LLM's so indeed for now we use it as Copilot and not the driver. Hopefully you can agree though its pretty random that we are still manually creating IAM policies and need to get accustomed with the thousands of different permissions :)
  - verdverm 3y ago
    We are actively working on reining in permissions, I would not call them "tight". It's just not a top 3 priority, though that is likely changing with the upcoming SOC2 efforts. I still don't see us reaching for LLMs to help us here. I'm not saying don't use them, just use them as an extra pair of eyes, mostly to catch errors rather than to drive and architect > get it that there is a lot of skepticism because of people hyping LLM's The skepticism is not from the hype, it's from experiencing LLM output personally. They are fine if the output can be fuzzy, like a blog post or a function signature, not so much if there is a specific and fragile target.
  - vasco 3y ago
    To add a plus one here, as soon as I learned there's LLMs involved this became a non starter to me. I'd rather have less granular policies than risk some LLM doing something crazy. I can justify to management that we have limited time for IAM and something was missed that we can fix / create tests / scans for after an incident. It's harder to explain that we chose a vendor that uses a non deterministic tool that can hallucinate for one of the most core security pieces of the puzzle.
- wg0 3y ago
  Absolutely not. Anywhere where accuracy, precision and safety matters, throwing LLMs in the mix is irresponsible IMHO or being too optimistic or possibly not understanding how these giant arrays of floating point numbers work or just hoping for the best. Similarly, LLMs used for SQL generation meant for business analytics is also a critical area where if numbers are wrong, it might lead to a business going bankrupt. For Prototype, fun exercise, sure go all in.
milkshakes 3y ago
that's a false dichotomy. there are approaches to this problem that are powered by neither humans nor LLMs -- see https://github.com/Netflix/Repokid https://github.com/Netflix/Repokid as an example
lijok 3y ago
> Research shows that 95% of the permissions granted to users aren't used These would be the "s3:*" and "Resources: *" scoped permissions I assume? I can't imagine users are explicitly typing out permissions, 95% of which are not relevant for the task. > which creates huge problems Such as? What is the material impact of a workflow or a user having too many permissions? > and is a reason for spending millions in security tools Are you claiming that overscoped IAM permissions alone are responsible for 1M+ security tooling bills in companies? Would you be willing to share information on which tools these are?
- kkapelon 3y ago
  > Such as? What is the material impact of a workflow or a user having too many permissions? Security obviously https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege
  - verdverm 3y ago
    That is the "theoretical" problem How many times have excess permissions "actually" been the problem... versus something like correct permissions with compromised credentials?
    - kkapelon 3y ago
      I am not a security expert by any means, but there are several stories of excess permissions that resulted in the security breaches. The last one I actually remember was here in HN, and I think it was about a bug bounty for Facebook where a QA system could affect production. The bug bounty person "broke" production by "breaking" in the QA system. By the way, I have no affiliation with slauth.io (just found them today as well). I just think that https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege is something good to follow in critical systems.
    - blincoln 3y ago
      It's hard to know with any kind of accuracy how often it comes up in real breaches, but we exploit it all the time to great effect in pen testing. A few examples I've seen repeatedly: * An AWS-hosted container/artifact/CI/CD application has an SSRF vulnerability that can be used to retrieve IAM instance credentials. Because micromanaging permissions is hard, and the application needs to access so much content in S3, spin up/down instances, etc. it has ec2:* and s3:. Unless the organization has created a separate AWS account for this platform specifically, it's probably game over at that point. An internet-facing MDM solution has a code execution vulnerability. Because the vendor didn't want to document all of the individual permissions it needs, the installation instructions specify that it should run as an account with Domain Admin permissions in AD. That is definitely game over for most organizations, because even systems that don't authenticate against AD are almost always accessed from systems that do. Micromanaging permissions is hard in a big organization. I saw it done well, years ago, in Active Directory, but it took several FTEs who were personally interested in the topic to set up and manage, and that was a traditional big business IT environment. In a startup-style free-for-all, good luck. I don't have an opinion either way on Slauth specifically, but something that generates IAM policies procedurally seems like a step in the right direction.
jmathai 3y ago
One challenge will be similar to self driving cars. The error / fatality rates need to be several orders of magnitude lower than for human operators for it to be acceptable.
candiddevmike 3y ago
AWS and GCP already provide tools to show excess permissions...
- verdverm 3y ago
  The pain there is often a pre-configured role with a slew of permissions was used and you actually need to craft a new role with the right permissions. I wrote some code once to fetch all those preconfigured role permissions and then present them in a more digestible way

jsploit 3y ago

> Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools. It'd potentially cost millions more to recover from a GPT-4 disaster.

verdverm 3y ago

I'm not your target user, I don't feel the priority on this problem even though our permissions are more permissive than we'd like. Thing is, to rein them in typically requires application changes. You cannot just sprinkle magic LLM dust on IAM and make things better. My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM and system architecture to know if what the LLM is saying is correct, so where does that leave us? I think they can be an extra pair of eyes, but not the driver. Still, there is a signal to noise problem that remains, due to the inherent hallucinations.

thehucklecat 3y ago
what kind of application changes are you thinking it would equire? my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).
- verdverm 3y ago
  Some scenarios 1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account. 2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production 3. You have to change human behaviors and habits (this is the real hard one). I've had to revert changes because the increased security blocked developers and they don't have time to adapt for the next deadline. 4. Getting parity in local development workflows is also challenging. How and where do you match vs except from IAM parity? 5. Should I give the current server access to a particular cloud service/resource or break out that particular function into a lambda and minimize the permissions there? You have to think through the implications of a breach and how/where you want to limit the blast radius. 6. This is probably obvious, but implementing application level controls, like API endpoint permissioning. IAM is not limited to cloud infra
  - DanielSlauth 3y ago
    The open-source project is a CLI you can put into your CI/CD so i think a pretty neat workflow where there should be less friction considering DevOps/security don't need to ping-pong on permissions.
    - verdverm 3y ago
      when you keep telling, you ain't selling ask questions to deepen your understanding > ...ping-pong... It was a scheduling problem rather than a decision problem. The impact radius is always more than you anticipate
DanielSlauth 3y ago
First of all its pretty awesome your permissions are very tight. You are definitely on the other side of the spectrum compared to the rest. I get it that there is a lot of skepticism because of people hyping LLM's so indeed for now we use it as Copilot and not the driver. Hopefully you can agree though its pretty random that we are still manually creating IAM policies and need to get accustomed with the thousands of different permissions :)
- verdverm 3y ago
  We are actively working on reining in permissions, I would not call them "tight". It's just not a top 3 priority, though that is likely changing with the upcoming SOC2 efforts. I still don't see us reaching for LLMs to help us here. I'm not saying don't use them, just use them as an extra pair of eyes, mostly to catch errors rather than to drive and architect > get it that there is a lot of skepticism because of people hyping LLM's The skepticism is not from the hype, it's from experiencing LLM output personally. They are fine if the output can be fuzzy, like a blog post or a function signature, not so much if there is a specific and fragile target.
- vasco 3y ago
  To add a plus one here, as soon as I learned there's LLMs involved this became a non starter to me. I'd rather have less granular policies than risk some LLM doing something crazy. I can justify to management that we have limited time for IAM and something was missed that we can fix / create tests / scans for after an incident. It's harder to explain that we chose a vendor that uses a non deterministic tool that can hallucinate for one of the most core security pieces of the puzzle.
wg0 3y ago
Absolutely not. Anywhere where accuracy, precision and safety matters, throwing LLMs in the mix is irresponsible IMHO or being too optimistic or possibly not understanding how these giant arrays of floating point numbers work or just hoping for the best. Similarly, LLMs used for SQL generation meant for business analytics is also a critical area where if numbers are wrong, it might lead to a business going bankrupt. For Prototype, fun exercise, sure go all in.

milkshakes 3y ago

that's a false dichotomy. there are approaches to this problem that are powered by neither humans nor LLMs -- see https://github.com/Netflix/Repokid https://github.com/Netflix/Repokid as an example

lijok 3y ago

> Research shows that 95% of the permissions granted to users aren't used These would be the "s3:*" and "Resources: *" scoped permissions I assume? I can't imagine users are explicitly typing out permissions, 95% of which are not relevant for the task. > which creates huge problems Such as? What is the material impact of a workflow or a user having too many permissions? > and is a reason for spending millions in security tools Are you claiming that overscoped IAM permissions alone are responsible for 1M+ security tooling bills in companies? Would you be willing to share information on which tools these are?

kkapelon 3y ago
> Such as? What is the material impact of a workflow or a user having too many permissions? Security obviously https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege
- verdverm 3y ago
  That is the "theoretical" problem How many times have excess permissions "actually" been the problem... versus something like correct permissions with compromised credentials?
  - kkapelon 3y ago
    I am not a security expert by any means, but there are several stories of excess permissions that resulted in the security breaches. The last one I actually remember was here in HN, and I think it was about a bug bounty for Facebook where a QA system could affect production. The bug bounty person "broke" production by "breaking" in the QA system. By the way, I have no affiliation with slauth.io (just found them today as well). I just think that https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege is something good to follow in critical systems.
  - blincoln 3y ago
    It's hard to know with any kind of accuracy how often it comes up in real breaches, but we exploit it all the time to great effect in pen testing. A few examples I've seen repeatedly: * An AWS-hosted container/artifact/CI/CD application has an SSRF vulnerability that can be used to retrieve IAM instance credentials. Because micromanaging permissions is hard, and the application needs to access so much content in S3, spin up/down instances, etc. it has ec2:* and s3:. Unless the organization has created a separate AWS account for this platform specifically, it's probably game over at that point. An internet-facing MDM solution has a code execution vulnerability. Because the vendor didn't want to document all of the individual permissions it needs, the installation instructions specify that it should run as an account with Domain Admin permissions in AD. That is definitely game over for most organizations, because even systems that don't authenticate against AD are almost always accessed from systems that do. Micromanaging permissions is hard in a big organization. I saw it done well, years ago, in Active Directory, but it took several FTEs who were personally interested in the topic to set up and manage, and that was a traditional big business IT environment. In a startup-style free-for-all, good luck. I don't have an opinion either way on Slauth specifically, but something that generates IAM policies procedurally seems like a step in the right direction.

jmathai 3y ago

One challenge will be similar to self driving cars. The error / fatality rates need to be several orders of magnitude lower than for human operators for it to be acceptable.

candiddevmike 3y ago

AWS and GCP already provide tools to show excess permissions...

verdverm 3y ago
The pain there is often a pre-configured role with a slew of permissions was used and you actually need to craft a new role with the right permissions. I wrote some code once to fetch all those preconfigured role permissions and then present them in a more digestible way