6 ms·

S3 security is a pain. S3 options are tricky to master, docs are beyond confusing. A common S3 use case is: Use a Bucket for Read-only static content, (js, htm

S3 security is a pain. S3 options are tricky to master, docs are beyond confusing. A common S3 use case is: Use a Bucket for Read-only static content, (js, html, imgs). From a dev perspective ideally it will work like a protected folder + Web Server, whereas the webserver will have read-only privileges, but reality is far more complicated. I can understand why was easy for Twillio to have this infosec issue, since we've done it ourselves many times when using S3.

ed25519FUUU 6y ago
Is it really a pain though? Maybe because I’ve been working with it since the beginning. Bucket ACL with NO permissions, then manage all permissions on an IAM role on the account. If it’s cross account then allow assuming to other accounts, but no reason to bother with the bucket ACL. Leave the bucket policy blank and you’ll never have to worry about an open bucket. Better yet make a deny rule to everything but a single role, and you don’t have to worry about rogue roles exposing the bucket. Perhaps some of the issue is that permission can be managed on both IAM and a bucket policy?
- deleted 6y ago
  [deleted]
- rafaelturk 6y ago
  This looks safe, and most importantly correct setup, but looks like it will not support Twillo use case
- throwlogon 6y ago
  What you described sounds like a pain.
  - ed25519FUUU 6y ago
    What’s the pain? You literally don’t do anything to the bucket policy. It’s intentionally blank. You already have a role in AWS that you use. Go into the console and say “this role can do S3 to this bucket” and you’re done.
- jfengel 6y ago
  I just started a toy project using S3, and at least to a novice user, it was a confusing, overwhelming array of options and linked pieces. It felt like being hit with a lot of advanced options all at once, with no obvious happy path for common cases. I'm sure an expert finds it all very natural. Security is always hard and has to be done right the first time. That's a domain that's always going to deter novices, and perhaps the only way forward is to hire an expert to ensure you're doing the right thing. Still... I've been programming for four decades and it felt a little odd to be quite so out of my depth. It's not the same as anything I've done before, but it felt like it shouldn't have been quite so bewildering.
- wjossey 6y ago
  This is the correct design pattern as of 2020. Roles + STS, and always be stingy when granting a role any permissions, both at the resource or API level. But, as others have said, the documentation at times is poor at best, and it can be easy to make mistakes. Some preventives every ops team can take are: 1. Never make manual permission changes in the console. Everything is done via SCM’d code. 2. All permission changes are peer reviewed. 3. Create logical separations of buckets so it’s more obvious what the permissions should be of a bucket. You can go so far as to have separate accounts that are for public uploads, separate accounts for public assets, etc. Avoid granting elevated permissions at a folder level, which is harder to understand as a developer without access to view the ACLs. Good luck everyone!
  - SV_BubbleTime 6y ago
    > Never make manual permission changes in the console. Everything is done via SCM’d code. The first time I heard “infrastructure as code” I thought it was some dumb new buzz words, but no, it’s good advice. Your cloud formation, serverless, terraform whatever should be stored in GIT a stack replace is how changes should be made. I consider the console as read-only.
    - nitrogen 6y ago
      IMO this is code/console dichotomy is a design flaw in most cloud offerings that will haunt the industry for a long time. The web console should be just another UI for editing the configuration in source control. As an example of why that's important, a UI can give you a warning and use affordances to guide beginners away from dangerous setups. In the case of S3, a slider for basic access level can use colors and show a list of objects that would become world readable or writable at a given level (for that matter why is world writable even an option?). A giant YAML file looks the same whether it's safe or dangerous. If you really don't want to use a UI, then maybe abstracting a linting layer that feeds the exact same warnings to the UI as to your CI system and text editor would help.
      - SV_BubbleTime 6y ago
- deleted 6y ago
  [deleted]
- antoncohen 6y ago
  It is way too easy to screw up S3 bucket permissions. I caught it once when a coworker told me they setup a bucket for testing that allowed employees to upload, I checked the permissions and it allowed the world to upload. It would be much harder make that mistake with Google Cloud Storage, at least in the web console. GCS also supports disabling bucket ACLs permanently at bucket creation time, and that is the option they recommend[1]. AWS does scan and alert for some public bucket issues. But with all the public S3 issues, I think it is clear there needs to be more prominent warnings in the AWS console, the command line, and with tools like Terraform. [1] https://cloud.google.com/storage/docs/access-control https://cloud.google.com/storage/docs/access-control
  - txcwpalpha 6y ago
    >GCS also supports disabling bucket ACLs permanently at bucket creation time, and that is the option they recommend[1]. S3 does too. There's an entire page during the creation wizard that is dedicated to blocking any and all public access, even causing the bucket to ignore ACLs or other settings that would otherwise expose the bucket publicly. All public access is disabled by default, and enabling it actually requires the user to actively uncheck 5 different checkboxes, each of which explains that unchecking it will open the bucket up to public access, and then even requires an additional attestation, in a large orange warning box, that says you acknowledge that the options you chose will result in the bucket being public. I'll be the first person to tell you that AWS is overcomplicated and hard to use, but this isn't that. IAM and bucket policies are a pain to work with, but if you screw up with those, the worse you will do is expose your bucket internally. But exposing a bucket publicly to the internet is an entirely different act, and there's not really any excuse for it other than just not reading the directions. However, neither this option in S3 nor the option you linked in GCS would have served Twilio's use case. They wanted their bucket to be publicly accessible, just not publicly writable. That's an entirely different access management issue.
- somehnguy 6y ago
  It is absolutely a pain. I'm using S3 for the first time on a project right now and literally every time I have to do anything in AWS I end up confused and scared that I'm leaving a wide open security vulnerability. The documentation is complete insanity to anyone just trying to accomplish what should be an extremely common use case in a reasonable amount of time. Before using AWS I was in the 'what morons!' camp whenever some company got breached via leaving things wide open on AWS. Now I'm in the 'ok I understand why its so common' camp.
  - txcwpalpha 6y ago
    The problem with S3 is the exact same problem with everything else in AWS. In their quest to be compatible and attractive to every single possible niche use case of the world's largest companies, they completely forgot the lone developer who just wants to upload a couple files. You'll even see this in their sales pitches. AWS will spend so much time talking about the most advanced use cases and how they are possible, but if you ask them a simple question about how to run a single bucket that doesn't also involve using all of their ML tools and global accelerator and cloudfront, they'll be blindsided. It's almost as if they consider the common use cases to be "too simple/obvious" and therefor they never bothered to create any documentation or put any thought into it.
    - pantulis 6y ago
      Your point is good, but one could suppose Twilio is not a "lone developer".
      - txcwpalpha 6y ago
        Certainly not, and in their layers of defense against this stuff, they should have had processes to train people, audit bucket policies, conduct penetration tests etc, and there's no excuse for them not having caught it before. But with that said, one of those layers of defense is also using tools that are easy to keep secure so that the vulnerability doesn't appear in the first place. While Twilio isn't "a lone developer", I wouldn't be surprised if the person who did create that bucket on behalf of Twilio was just a lone developer fumbling around in the console. And again, that's no excuse, but it still is an area for improvement.
- meritt 6y ago
  I have external partners who provide their own principal and I provision a subset of bucket access at the bucket policy level. Is there a better way to do this that's more manageable? The end users are using software or python code where they need to gain access via [access, secret, bucket, prefix/key].
merb 6y ago
actually aws changed the default lately and added more options to really see what you configured. in the past, there was no indicator if s3 is private only oder public
mcraiha 6y ago
Is there a service that one could use to check if their bucket accepts public uploads or public ACL changes?
- mwarkentin 6y ago
  Aws S3 surfaces this info now, as well as allowing you to enforce that they’re disabled at the bucket or account level (public access blocks).
harrisonjackson 6y ago
Agreed. Misconfigured s3 buckets have to be in the top 1% of of causes for data leaks, hacking, etc security issues in the past decade.

ed25519FUUU 6y ago

Is it really a pain though? Maybe because I’ve been working with it since the beginning. Bucket ACL with NO permissions, then manage all permissions on an IAM role on the account. If it’s cross account then allow assuming to other accounts, but no reason to bother with the bucket ACL. Leave the bucket policy blank and you’ll never have to worry about an open bucket. Better yet make a deny rule to everything but a single role, and you don’t have to worry about rogue roles exposing the bucket. Perhaps some of the issue is that permission can be managed on both IAM and a bucket policy?

deleted 6y ago
[deleted]
rafaelturk 6y ago
This looks safe, and most importantly correct setup, but looks like it will not support Twillo use case
throwlogon 6y ago
What you described sounds like a pain.
- ed25519FUUU 6y ago
  What’s the pain? You literally don’t do anything to the bucket policy. It’s intentionally blank. You already have a role in AWS that you use. Go into the console and say “this role can do S3 to this bucket” and you’re done.
jfengel 6y ago
I just started a toy project using S3, and at least to a novice user, it was a confusing, overwhelming array of options and linked pieces. It felt like being hit with a lot of advanced options all at once, with no obvious happy path for common cases. I'm sure an expert finds it all very natural. Security is always hard and has to be done right the first time. That's a domain that's always going to deter novices, and perhaps the only way forward is to hire an expert to ensure you're doing the right thing. Still... I've been programming for four decades and it felt a little odd to be quite so out of my depth. It's not the same as anything I've done before, but it felt like it shouldn't have been quite so bewildering.
wjossey 6y ago
This is the correct design pattern as of 2020. Roles + STS, and always be stingy when granting a role any permissions, both at the resource or API level. But, as others have said, the documentation at times is poor at best, and it can be easy to make mistakes. Some preventives every ops team can take are: 1. Never make manual permission changes in the console. Everything is done via SCM’d code. 2. All permission changes are peer reviewed. 3. Create logical separations of buckets so it’s more obvious what the permissions should be of a bucket. You can go so far as to have separate accounts that are for public uploads, separate accounts for public assets, etc. Avoid granting elevated permissions at a folder level, which is harder to understand as a developer without access to view the ACLs. Good luck everyone!
- SV_BubbleTime 6y ago
  > Never make manual permission changes in the console. Everything is done via SCM’d code. The first time I heard “infrastructure as code” I thought it was some dumb new buzz words, but no, it’s good advice. Your cloud formation, serverless, terraform whatever should be stored in GIT a stack replace is how changes should be made. I consider the console as read-only.
  - nitrogen 6y ago
    IMO this is code/console dichotomy is a design flaw in most cloud offerings that will haunt the industry for a long time. The web console should be just another UI for editing the configuration in source control. As an example of why that's important, a UI can give you a warning and use affordances to guide beginners away from dangerous setups. In the case of S3, a slider for basic access level can use colors and show a list of objects that would become world readable or writable at a given level (for that matter why is world writable even an option?). A giant YAML file looks the same whether it's safe or dangerous. If you really don't want to use a UI, then maybe abstracting a linting layer that feeds the exact same warnings to the UI as to your CI system and text editor would help.
    - SV_BubbleTime 6y ago
deleted 6y ago
[deleted]
antoncohen 6y ago
It is way too easy to screw up S3 bucket permissions. I caught it once when a coworker told me they setup a bucket for testing that allowed employees to upload, I checked the permissions and it allowed the world to upload. It would be much harder make that mistake with Google Cloud Storage, at least in the web console. GCS also supports disabling bucket ACLs permanently at bucket creation time, and that is the option they recommend[1]. AWS does scan and alert for some public bucket issues. But with all the public S3 issues, I think it is clear there needs to be more prominent warnings in the AWS console, the command line, and with tools like Terraform. [1] https://cloud.google.com/storage/docs/access-control https://cloud.google.com/storage/docs/access-control
- txcwpalpha 6y ago
  >GCS also supports disabling bucket ACLs permanently at bucket creation time, and that is the option they recommend[1]. S3 does too. There's an entire page during the creation wizard that is dedicated to blocking any and all public access, even causing the bucket to ignore ACLs or other settings that would otherwise expose the bucket publicly. All public access is disabled by default, and enabling it actually requires the user to actively uncheck 5 different checkboxes, each of which explains that unchecking it will open the bucket up to public access, and then even requires an additional attestation, in a large orange warning box, that says you acknowledge that the options you chose will result in the bucket being public. I'll be the first person to tell you that AWS is overcomplicated and hard to use, but this isn't that. IAM and bucket policies are a pain to work with, but if you screw up with those, the worse you will do is expose your bucket internally. But exposing a bucket publicly to the internet is an entirely different act, and there's not really any excuse for it other than just not reading the directions. However, neither this option in S3 nor the option you linked in GCS would have served Twilio's use case. They wanted their bucket to be publicly accessible, just not publicly writable. That's an entirely different access management issue.
somehnguy 6y ago
It is absolutely a pain. I'm using S3 for the first time on a project right now and literally every time I have to do anything in AWS I end up confused and scared that I'm leaving a wide open security vulnerability. The documentation is complete insanity to anyone just trying to accomplish what should be an extremely common use case in a reasonable amount of time. Before using AWS I was in the 'what morons!' camp whenever some company got breached via leaving things wide open on AWS. Now I'm in the 'ok I understand why its so common' camp.
- txcwpalpha 6y ago
  The problem with S3 is the exact same problem with everything else in AWS. In their quest to be compatible and attractive to every single possible niche use case of the world's largest companies, they completely forgot the lone developer who just wants to upload a couple files. You'll even see this in their sales pitches. AWS will spend so much time talking about the most advanced use cases and how they are possible, but if you ask them a simple question about how to run a single bucket that doesn't also involve using all of their ML tools and global accelerator and cloudfront, they'll be blindsided. It's almost as if they consider the common use cases to be "too simple/obvious" and therefor they never bothered to create any documentation or put any thought into it.
  - pantulis 6y ago
    Your point is good, but one could suppose Twilio is not a "lone developer".
    - txcwpalpha 6y ago
      Certainly not, and in their layers of defense against this stuff, they should have had processes to train people, audit bucket policies, conduct penetration tests etc, and there's no excuse for them not having caught it before. But with that said, one of those layers of defense is also using tools that are easy to keep secure so that the vulnerability doesn't appear in the first place. While Twilio isn't "a lone developer", I wouldn't be surprised if the person who did create that bucket on behalf of Twilio was just a lone developer fumbling around in the console. And again, that's no excuse, but it still is an area for improvement.
meritt 6y ago
I have external partners who provide their own principal and I provision a subset of bucket access at the bucket policy level. Is there a better way to do this that's more manageable? The end users are using software or python code where they need to gain access via [access, secret, bucket, prefix/key].

merb 6y ago

actually aws changed the default lately and added more options to really see what you configured. in the past, there was no indicator if s3 is private only oder public

mcraiha 6y ago

Is there a service that one could use to check if their bucket accepts public uploads or public ACL changes?

mwarkentin 6y ago
Aws S3 surfaces this info now, as well as allowing you to enforce that they’re disabled at the bucket or account level (public access blocks).

harrisonjackson 6y ago

Agreed. Misconfigured s3 buckets have to be in the top 1% of of causes for data leaks, hacking, etc security issues in the past decade.