4 ms·

This is a useful and much needed update. Know lots of stories where people got caught off guard by ACLs setting things public that they thought were private. U

This is a useful and much needed update. Know lots of stories where people got caught off guard by ACLs setting things public that they thought were private. Usual scenario is that the bucket is set as “private” and you think it’s private but realize some app or script has set the ACL for objects to “public.”

privateSFacct 8y ago
"got caught off guard by ACLs setting things public that they thought were private." A lot of these stories are written in this way where Amazon's ACL's "set thing public" when the user wanted things private. To add a little reality here: Amazon defaults are generally to private. You get no network ingress to an instance by default (zero). You have no username/password login by default (zero) - you need to use SSH with a public key. S3 buckets are resource owner and aws account owner only to start. My one issue - I wish they made buckets with no permissions by default, make the account owner add themselves or resource owner add themselves. What is happening is despite LOTS of docs out there, users find it easier during development, quick integration to do things like WORLD readable. This is BY FAR the most common issue, USERS (not amazon) setting things to totally public by choice. I've had folks tell say: * It will only be public for a few days / short period of time while we work out X (once everything is used to public access it actually is HARDER to switch to secure access). * I can't be bothered to learn IAM permissions (one of the most useful features AWS has for example relative to a place like google). I like this new thing not because amazon is at fault, but because users are super lazy, and now some annoying "admin" can block them from making things totally public when they want to share with a partner, separate project etc. So a good feature. Now make S3 bucket's with NO permissions by default (not even creator / account owner) so folks can learn a bit of AWS right away to get going.
- jrochkind1 8y ago
  Agreed. ACL's can be set on a per-object basis. So you might have set up the bucket thinking you set it up as "non-public", but them some software sets objects to public when it stores them. And we all know about misbehaving software and bugs. And how would you know that had happened? I guess you'd have to run something that monitored all the ACLs? (And you'd pay for those monitoring operations, although probably not a significant cost). It makes a _lot_ of sense to be able to tell a bucket "No public access allowed in here". Which I understand is included in this new feature set? That alone is only sensible.
  - privateSFacct 8y ago
    This is actually very common (software setting public access). I think it's to reduce support burden on software developers. See mongo with no access control at all, lots of plugins and apps that might serve something public go to ALL public etc etc. They've had free bucket permission check tools out I think since at least February as well. I think that will continue.
    - jrochkind1 8y ago
      Nice, I didn't know about the free bucket permission check tools, that's good. Looks like here's the announcement: https://aws.amazon.com/about-aws/whats-new/2018/02/aws-trusted-advisors-s3-bucket-permissions-check-is-now-free/ https://aws.amazon.com/about-aws/whats-new/2018/02/aws-trust... Sounds like you can still only automate it (using AWS dashboard anyway) as "Business and Enterprise support customers." But yeah, simply being able to set a bucket as "never allow public access in here" makes a _lot_ of sense, an easy way to deal with "misbehaving" software components (whether the developers who wrote them believed it was misbehavior or not). In most of the web software I write these days, I make _nothing_ public in S3, but provide an endpoint that will HTTP redirect to a signed URL to grant access. It's a bit of extra work (cpu cycles for the software that is, relatively trivial for the developer), but makes it easier to keep track of what I've granted access to how and when.
- dsfyu404ed 8y ago
  I agree People blame the power tool manufacture after they get hurt because they wired open the guard, taped down the safety and then developed a habit of holding the saw in one hand and the work in the other. Some people just can't accept that they dug their own hole even when it they clearly went out of their way to find the shovel.
- vegardx 8y ago
  A good habit to get into is to create a bucket policy to explicitly deny these actions.
- dylan604 8y ago
  This is where I have actually come to appreciate SELinux. A newer dev comes along and makes a folder in the document root to 777 thinking that will solve their problems. SELinux still needs to explicitly allow that folder to be writable. That slows them down long enough to come find me, and then we get to have the proper conversation about what needs to happen. A folder in the document root with 777 scares the crap out of me
  - morpheuskafka 8y ago
    Do you know of any good intro to SELinux guides? I'm hoping to use it to lock down webroots to prevent other users from modifying them even if the user messes up all the permissions via SFTP.
    - dylan604 8y ago
      Sadly, no. I only play a sysadmin on TV ;-) I don't fully feel like I have groked SELinux, but when things behave unexpectedly, it takes me less and less time to remember to check if SELinux is involved. I have at least come to accept that it is there to help, and it has saved me from doing some really dumb things. Things to know is that there is a specific setting to allow httpd to write to a folder. There is a way to list files `ls -Z` that shows you the SELinux Context for files/folders. httpd error_log entries will just give permission denied errors, but if you feel like the perms are correct, SELinux will probably be why you're getting denied. That's how I started learning. One error at a time.

privateSFacct 8y ago

"got caught off guard by ACLs setting things public that they thought were private." A lot of these stories are written in this way where Amazon's ACL's "set thing public" when the user wanted things private. To add a little reality here: Amazon defaults are generally to private. You get no network ingress to an instance by default (zero). You have no username/password login by default (zero) - you need to use SSH with a public key. S3 buckets are resource owner and aws account owner only to start. My one issue - I wish they made buckets with no permissions by default, make the account owner add themselves or resource owner add themselves. What is happening is despite LOTS of docs out there, users find it easier during development, quick integration to do things like WORLD readable. This is BY FAR the most common issue, USERS (not amazon) setting things to totally public by choice. I've had folks tell say: * It will only be public for a few days / short period of time while we work out X (once everything is used to public access it actually is HARDER to switch to secure access). * I can't be bothered to learn IAM permissions (one of the most useful features AWS has for example relative to a place like google). I like this new thing not because amazon is at fault, but because users are super lazy, and now some annoying "admin" can block them from making things totally public when they want to share with a partner, separate project etc. So a good feature. Now make S3 bucket's with NO permissions by default (not even creator / account owner) so folks can learn a bit of AWS right away to get going.

jrochkind1 8y ago
Agreed. ACL's can be set on a per-object basis. So you might have set up the bucket thinking you set it up as "non-public", but them some software sets objects to public when it stores them. And we all know about misbehaving software and bugs. And how would you know that had happened? I guess you'd have to run something that monitored all the ACLs? (And you'd pay for those monitoring operations, although probably not a significant cost). It makes a _lot_ of sense to be able to tell a bucket "No public access allowed in here". Which I understand is included in this new feature set? That alone is only sensible.
- privateSFacct 8y ago
  This is actually very common (software setting public access). I think it's to reduce support burden on software developers. See mongo with no access control at all, lots of plugins and apps that might serve something public go to ALL public etc etc. They've had free bucket permission check tools out I think since at least February as well. I think that will continue.
  - jrochkind1 8y ago
    Nice, I didn't know about the free bucket permission check tools, that's good. Looks like here's the announcement: https://aws.amazon.com/about-aws/whats-new/2018/02/aws-trusted-advisors-s3-bucket-permissions-check-is-now-free/ https://aws.amazon.com/about-aws/whats-new/2018/02/aws-trust... Sounds like you can still only automate it (using AWS dashboard anyway) as "Business and Enterprise support customers." But yeah, simply being able to set a bucket as "never allow public access in here" makes a _lot_ of sense, an easy way to deal with "misbehaving" software components (whether the developers who wrote them believed it was misbehavior or not). In most of the web software I write these days, I make _nothing_ public in S3, but provide an endpoint that will HTTP redirect to a signed URL to grant access. It's a bit of extra work (cpu cycles for the software that is, relatively trivial for the developer), but makes it easier to keep track of what I've granted access to how and when.
dsfyu404ed 8y ago
I agree People blame the power tool manufacture after they get hurt because they wired open the guard, taped down the safety and then developed a habit of holding the saw in one hand and the work in the other. Some people just can't accept that they dug their own hole even when it they clearly went out of their way to find the shovel.
vegardx 8y ago
A good habit to get into is to create a bucket policy to explicitly deny these actions.
dylan604 8y ago
This is where I have actually come to appreciate SELinux. A newer dev comes along and makes a folder in the document root to 777 thinking that will solve their problems. SELinux still needs to explicitly allow that folder to be writable. That slows them down long enough to come find me, and then we get to have the proper conversation about what needs to happen. A folder in the document root with 777 scares the crap out of me
- morpheuskafka 8y ago
  Do you know of any good intro to SELinux guides? I'm hoping to use it to lock down webroots to prevent other users from modifying them even if the user messes up all the permissions via SFTP.
  - dylan604 8y ago
    Sadly, no. I only play a sysadmin on TV ;-) I don't fully feel like I have groked SELinux, but when things behave unexpectedly, it takes me less and less time to remember to check if SELinux is involved. I have at least come to accept that it is there to help, and it has saved me from doing some really dumb things. Things to know is that there is a specific setting to allow httpd to write to a folder. There is a way to list files `ls -Z` that shows you the SELinux Context for files/folders. httpd error_log entries will just give permission denied errors, but if you feel like the perms are correct, SELinux will probably be why you're getting denied. That's how I started learning. One error at a time.