4 ms·

In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures

In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures. There's no actual control improvements here, just "we'll follow our procedures better next time" which imo is effectively doing nothing. Also this is really lacking in detail about how it was determined that no PII was accessed. What audit logs were checked? Where was this data stored? Overall this is a super disappointing postmortem...

nerdjon 1y ago
> In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures. I am curious what preventative measures you expect in this situation? To my knowledge it is not actually possible to disable the root account. They also had it restricted to only 3 people with MFA which also seems pretty reasonable. It is not unheard of that there could be a situation where your ability to login through normal means (like lets say it relies on Okta and Okta goes down) and you need to get into the account, root may be your only option in a disaster situation. Given this was specifically for oncall someone having that makes sense. Not saying there were not failures because there clearly are, but there have been times I have had to use root when I had no other option to get into an account.
- flumpcakes 1y ago
  I'm now questioning my sanity but I thought you could disable login for the root account in AWS.
  - the_mitsuhiko 1y ago
    Since there are certain operations that can only be done with the root account, there is no way to disable access to it.
    - jhealy 1y ago
      Since 2024 you can disable the root credentials on all accounts except the Organization management account: https://aws.amazon.com/blogs/aws/centrally-managing-root-access-for-customers-using-aws-organizations/ https://aws.amazon.com/blogs/aws/centrally-managing-root-acc... I don't think the post mortem details whether the root access was on the org management account or an org member account.
      - the_mitsuhiko 1y ago
        Oh wow. I completely missed that change.
- oneplane 1y ago
  You don't need the root account, unless you need to bypass all policies. In such a scenario, you a use the root access reset flow instead, reducing standing access. As for other flows (break glass, non-SSO etc), that can all be handled using IAM users. You'd normally use SAML to assume a role, but when SSO is down you'd use your fallback IAM user and then assume the role you need. As for how you disable the root account: solo accounts can't, but you can still prevent use/mis-use by setting a random long password and not writing it down anywhere. In an Org, the org can disable root on member accounts.
  - nerdjon 1y ago
    To me that sounds like security by obscurity not actual security. If you have the ability to go through the reset flow than then why is that much different than the username and password being available to a limited sets of users. That would not have prevented this from happening if the determination was made that all 3 of these users need the ability to possibly get into root. As far as having an IAM user, I fail to see how that is actually that much better. You still have a user sitting there with long running credentials that need to be saved somewhere that is outside of how you normally access AWS. Meaning it is also something that could be easily missed if someone left. Sure yes you could argue that the root user and that IAM user would have drastically different permissions, but the core problem would still exist. But then you are adding another account(s) on top of the root account that must exist that you now need to worry about. Regardless of the option you take, the root of the problem they had was 2fold. Not only did they not have alerts on the usage of the root account (which they would still need if they switched to having long running IAM users instead, but now they would also need to monitor root since that reset flow exists) and their offboarding workflow did not properly rotate that password, which a similar problem would also exist with a long running IAM user to delete that account. At the end of the day there is not a perfect solution to this problem, but I think just saying that you would never use root is ignoring several other issues that don't go away just by not using root.
    - zrail 1y ago
      Resetting the root password requires proving access to the email address associated with the root account. It also leaves a massive papertrail.
brian_cunnie 1y ago
Sometimes I log into the root account to see the billing information. I created an "administrator" account, but apparently it can't see the billing information, including the very-important amount of remaining cloud credits. Maybe I could spend time fiddling with IAM to get the right privileges, but I have more pressing tasks. And besides, on my personal AWS account I only log in with the root account.

nerdjon 1y ago

> In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures. I am curious what preventative measures you expect in this situation? To my knowledge it is not actually possible to disable the root account. They also had it restricted to only 3 people with MFA which also seems pretty reasonable. It is not unheard of that there could be a situation where your ability to login through normal means (like lets say it relies on Okta and Okta goes down) and you need to get into the account, root may be your only option in a disaster situation. Given this was specifically for oncall someone having that makes sense. Not saying there were not failures because there clearly are, but there have been times I have had to use root when I had no other option to get into an account.

flumpcakes 1y ago
I'm now questioning my sanity but I thought you could disable login for the root account in AWS.
- the_mitsuhiko 1y ago
  Since there are certain operations that can only be done with the root account, there is no way to disable access to it.
  - jhealy 1y ago
    Since 2024 you can disable the root credentials on all accounts except the Organization management account: https://aws.amazon.com/blogs/aws/centrally-managing-root-access-for-customers-using-aws-organizations/ https://aws.amazon.com/blogs/aws/centrally-managing-root-acc... I don't think the post mortem details whether the root access was on the org management account or an org member account.
    - the_mitsuhiko 1y ago
      Oh wow. I completely missed that change.
oneplane 1y ago
You don't need the root account, unless you need to bypass all policies. In such a scenario, you a use the root access reset flow instead, reducing standing access. As for other flows (break glass, non-SSO etc), that can all be handled using IAM users. You'd normally use SAML to assume a role, but when SSO is down you'd use your fallback IAM user and then assume the role you need. As for how you disable the root account: solo accounts can't, but you can still prevent use/mis-use by setting a random long password and not writing it down anywhere. In an Org, the org can disable root on member accounts.
- nerdjon 1y ago
  To me that sounds like security by obscurity not actual security. If you have the ability to go through the reset flow than then why is that much different than the username and password being available to a limited sets of users. That would not have prevented this from happening if the determination was made that all 3 of these users need the ability to possibly get into root. As far as having an IAM user, I fail to see how that is actually that much better. You still have a user sitting there with long running credentials that need to be saved somewhere that is outside of how you normally access AWS. Meaning it is also something that could be easily missed if someone left. Sure yes you could argue that the root user and that IAM user would have drastically different permissions, but the core problem would still exist. But then you are adding another account(s) on top of the root account that must exist that you now need to worry about. Regardless of the option you take, the root of the problem they had was 2fold. Not only did they not have alerts on the usage of the root account (which they would still need if they switched to having long running IAM users instead, but now they would also need to monitor root since that reset flow exists) and their offboarding workflow did not properly rotate that password, which a similar problem would also exist with a long running IAM user to delete that account. At the end of the day there is not a perfect solution to this problem, but I think just saying that you would never use root is ignoring several other issues that don't go away just by not using root.
  - zrail 1y ago
    Resetting the root password requires proving access to the email address associated with the root account. It also leaves a massive papertrail.

brian_cunnie 1y ago

Sometimes I log into the root account to see the billing information. I created an "administrator" account, but apparently it can't see the billing information, including the very-important amount of remaining cloud credits. Maybe I could spend time fiddling with IAM to get the right privileges, but I have more pressing tasks. And besides, on my personal AWS account I only log in with the root account.