4 ms·

I'm starting to think we should just not allow developers to use s3 anymore. Despite Amazon plastering it with warnings, these breaches keep happening. That or

I'm starting to think we should just not allow developers to use s3 anymore. Despite Amazon plastering it with warnings, these breaches keep happening. That or we need to start fining heavily for breaches.

ceejayoz 5y ago
> I'm starting to think we should just not allow developers to use s3 anymore. I don't know that the alternatives would be much better. People have inadvertently made folders on Apache wide open for decades now. AWS does more than average to combat it, I'd say. E-mail notifications, default configs, scary warnings, etc.
figassis 5y ago
I still don't get the s3 breach issue. I mean, buckets are private by default. Why would anyone take any action to open an s3 bucket that stores logs? If you do nothing you're already halfway there. Or is there a failure mode that I'm not aware of?
- zo1 5y ago
  My take: S3 buckets have an "air" or "impression" of being easy to "host" publicly available "things". It's even one of their selling points, IIRC. I.e. hosting a PWA/index.html website on a bucket.
- dragonwriter 5y ago
  I'm guessing that there are lots of (public and internal) tools, documented workflows, etc., dealing with S3 buckets that implement very bad security defaults because it streamlines getting something apparently functional and working on it in a dev environment, even though AWS itself has secure defaults, and using those easy-for-dev approaches with live PII of otherwise critical data is a recipe for disaster. People often aren’t starting with the AWS defaults, they are starting with an IaC (Cloud formation, CDK, Terraform) template they got from some other project.
- paranoidrobot 5y ago
  I don't know about this specific case, but sometimes trying to get log delivery from AWS services has been finicky and annoying to set up correctly. Different Account IDs for regions and products. There's also some configurations that require you to open access to the S3 bucket to get Cloudfront to point at it, because their Origin Access ID configuration doesn't work for all scenarios. I know some of these things were sorted, but I don't have access to the old configurations to prompt me to remember what they are.
selecsosi 5y ago
You can enable public file ACL access restriction at the account level and issue authenticated/signed links for file access if required for general consumption. The idea that your data layer shouldn't be open to unauthenticated read from the public shouldn't be new to people but persists with somewhat "easy" buttons to enable behaviors. For static sites or other public access required files setting up cloudfront with an authenticated origin pull is pretty straightforward and in our case we use a terraform module to provision and secure the bucket/distribution. I think this come when you get dev/biz users with console access who are trying to "just get it done" when you are dealing with highly confidential or sensitive data, it's a recipe for leak.
e2le 5y ago
I suspect there is a degree of "not-caring" and/or incompetence among at least some of these developers. Banning their use of s3 likely isn't a solution, it's a culture problem.
tomc1985 5y ago
It seems like providing sane, secure defaults would be more effective. But that would probably destroy the onboarding experience, how many tech firms release big products that should have authentication but really don't because of "ease of use"? (Here's looking at you, ElasticSearch. Did they finally fix this?)
- ghostpepper 5y ago
  when I installed an ELK stack recently it took a fair bit of effort to get it secured but at least there were big scary warnings that it wasn't
- vorpalhex 5y ago
  In s3 you not only have to mark the bucket as public, but also toggle a safety switch allowing the bucket to be made public, and you have to write an ACL policy allowing public access, and you will get a monthly email from AWS reminding you that this bucket is setup for public access. Oh, and the bucket has the word "public" next to it. Like, what more can AWS do? Require a blood sacrifice? Call your boss?
  - isbvhodnvemrwvn 5y ago
    Also there's an account-level switch that can prevent any bucket from going public.
  - tomc1985 5y ago
    Good, because the last time I used s3 (years ago) IIRC it seemed like public was the default
sdoering 5y ago
If I don't secure any resources on AWS, even for a one off test dummy without any real data in it I receive a barrage of eMails from CIO. So does my technical studio lead. If we don't react within 24 hours the technical MD of our org will be on the receiving end of these emails as well. So within the org I am part of it is actually absolutely clear what the basic security configuration is. But it is also such a hassle, that I at least (being a data analyst sometimes in need of a short one off project for testing) just don't use it anymore for anything ephemeral (living few days at most) without containing any real world data. Securing the AWS instance would more often than not actually take nearly as much time as setting up a proof of concept project for a demo itself. More often than not that is only replicating an existing poc and changing a few images, fonts and a bit of text to make it look more like the client's pages/corporate identity. Then a short demo in a meeting and after that a tear down. Worst case these demos are live for 3 days. So security as well as compliance add 50 - 100% of effort in these cases. Making POCs on AWS just not worth the effort. As much as I understand that anything more than a poc needs to be secured I would wish for a way of communicating that some things are just so temporary as to not warrant this overkill. But as this could be abused, there just isn't. So tl;dr: Orga can ensure security and best practices in AWS if the invest to monitor and check automatically. But that does have other consequences as well. [Edit] : typos

ceejayoz 5y ago

> I'm starting to think we should just not allow developers to use s3 anymore. I don't know that the alternatives would be much better. People have inadvertently made folders on Apache wide open for decades now. AWS does more than average to combat it, I'd say. E-mail notifications, default configs, scary warnings, etc.

figassis 5y ago

I still don't get the s3 breach issue. I mean, buckets are private by default. Why would anyone take any action to open an s3 bucket that stores logs? If you do nothing you're already halfway there. Or is there a failure mode that I'm not aware of?

zo1 5y ago
My take: S3 buckets have an "air" or "impression" of being easy to "host" publicly available "things". It's even one of their selling points, IIRC. I.e. hosting a PWA/index.html website on a bucket.
dragonwriter 5y ago
I'm guessing that there are lots of (public and internal) tools, documented workflows, etc., dealing with S3 buckets that implement very bad security defaults because it streamlines getting something apparently functional and working on it in a dev environment, even though AWS itself has secure defaults, and using those easy-for-dev approaches with live PII of otherwise critical data is a recipe for disaster. People often aren’t starting with the AWS defaults, they are starting with an IaC (Cloud formation, CDK, Terraform) template they got from some other project.
paranoidrobot 5y ago
I don't know about this specific case, but sometimes trying to get log delivery from AWS services has been finicky and annoying to set up correctly. Different Account IDs for regions and products. There's also some configurations that require you to open access to the S3 bucket to get Cloudfront to point at it, because their Origin Access ID configuration doesn't work for all scenarios. I know some of these things were sorted, but I don't have access to the old configurations to prompt me to remember what they are.

selecsosi 5y ago

You can enable public file ACL access restriction at the account level and issue authenticated/signed links for file access if required for general consumption. The idea that your data layer shouldn't be open to unauthenticated read from the public shouldn't be new to people but persists with somewhat "easy" buttons to enable behaviors. For static sites or other public access required files setting up cloudfront with an authenticated origin pull is pretty straightforward and in our case we use a terraform module to provision and secure the bucket/distribution. I think this come when you get dev/biz users with console access who are trying to "just get it done" when you are dealing with highly confidential or sensitive data, it's a recipe for leak.

e2le 5y ago

I suspect there is a degree of "not-caring" and/or incompetence among at least some of these developers. Banning their use of s3 likely isn't a solution, it's a culture problem.

tomc1985 5y ago

It seems like providing sane, secure defaults would be more effective. But that would probably destroy the onboarding experience, how many tech firms release big products that should have authentication but really don't because of "ease of use"? (Here's looking at you, ElasticSearch. Did they finally fix this?)

ghostpepper 5y ago
when I installed an ELK stack recently it took a fair bit of effort to get it secured but at least there were big scary warnings that it wasn't
vorpalhex 5y ago
In s3 you not only have to mark the bucket as public, but also toggle a safety switch allowing the bucket to be made public, and you have to write an ACL policy allowing public access, and you will get a monthly email from AWS reminding you that this bucket is setup for public access. Oh, and the bucket has the word "public" next to it. Like, what more can AWS do? Require a blood sacrifice? Call your boss?
- isbvhodnvemrwvn 5y ago
  Also there's an account-level switch that can prevent any bucket from going public.
- tomc1985 5y ago
  Good, because the last time I used s3 (years ago) IIRC it seemed like public was the default

sdoering 5y ago

If I don't secure any resources on AWS, even for a one off test dummy without any real data in it I receive a barrage of eMails from CIO. So does my technical studio lead. If we don't react within 24 hours the technical MD of our org will be on the receiving end of these emails as well. So within the org I am part of it is actually absolutely clear what the basic security configuration is. But it is also such a hassle, that I at least (being a data analyst sometimes in need of a short one off project for testing) just don't use it anymore for anything ephemeral (living few days at most) without containing any real world data. Securing the AWS instance would more often than not actually take nearly as much time as setting up a proof of concept project for a demo itself. More often than not that is only replicating an existing poc and changing a few images, fonts and a bit of text to make it look more like the client's pages/corporate identity. Then a short demo in a meeting and after that a tear down. Worst case these demos are live for 3 days. So security as well as compliance add 50 - 100% of effort in these cases. Making POCs on AWS just not worth the effort. As much as I understand that anything more than a poc needs to be secured I would wish for a way of communicating that some things are just so temporary as to not warrant this overkill. But as this could be abused, there just isn't. So tl;dr: Orga can ensure security and best practices in AWS if the invest to monitor and check automatically. But that does have other consequences as well. [Edit] : typos