10 ms·
Amazon S3 Block Public Access: Protection for Your Accounts and Buckets
- fredley 8y agoI could have had this a few weeks ago, when I realised popular S3 integration tool `django-storages` sets all objects' ACL as public-read by default: https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html https://django-storages.readthedocs.io/en/latest/backends/am...
- cbg0 8y agoCan't really fault Amazon or the Django library for you not reading the docs.
- geofft 8y agoI am 100% willing to fault software for bad defaults regardless of how they document those defaults.
- nscalf 8y agoAbsolutely agree. Telling people of your bad design is still bad. Especially when "telling people" happens in a flood of other noise, like one line in documentation on a whole tool.
- Alex3917 8y ago> Telling people of your bad design is still bad. IIRC the person who took over maintenance and added that warning isn't the one who created the tool.
- fredley 8y agoDefault-insecure is bad design almost anywhere I can think of (and Django Storages is changing their default in the next version - they realised it's probably not best as it is).
- jon_richards 8y agoThat got fixed. Default behaviour is now to use the bucket's default.
- code4tee 8y agoThis is a useful and much needed update. Know lots of stories where people got caught off guard by ACLs setting things public that they thought were private. Usual scenario is that the bucket is set as “private” and you think it’s private but realize some app or script has set the ACL for objects to “public.”
- privateSFacct 8y ago"got caught off guard by ACLs setting things public that they thought were private." A lot of these stories are written in this way where Amazon's ACL's "set thing public" when the user wanted things private. To add a little reality here: Amazon defaults are generally to private. You get no network ingress to an instance by default (zero). You have no username/password login by default (zero) - you need to use SSH with a public key. S3 buckets are resource owner and aws account owner only to start. My one issue - I wish they made buckets with no permissions by default, make the account owner add themselves or resource owner add themselves. What is happening is despite LOTS of docs out there, users find it easier during development, quick integration to do things like WORLD readable. This is BY FAR the most common issue, USERS (not amazon) setting things to totally public by choice. I've had folks tell say: * It will only be public for a few days / short period of time while we work out X (once everything is used to public access it actually is HARDER to switch to secure access). * I can't be bothered to learn IAM permissions (one of the most useful features AWS has for example relative to a place like google). I like this new thing not because amazon is at fault, but because users are super lazy, and now some annoying "admin" can block them from making things totally public when they want to share with a partner, separate project etc. So a good feature. Now make S3 bucket's with NO permissions by default (not even creator / account owner) so folks can learn a bit of AWS right away to get going.
- jrochkind1 8y agoAgreed. ACL's can be set on a per-object basis. So you might have set up the bucket thinking you set it up as "non-public", but them some software sets objects to public when it stores them. And we all know about misbehaving software and bugs. And how would you know that had happened? I guess you'd have to run something that monitored all the ACLs? (And you'd pay for those monitoring operations, although probably not a significant cost). It makes a _lot_ of sense to be able to tell a bucket "No public access allowed in here". Which I understand is included in this new feature set? That alone is only sensible.
- ams6110 8y agoIt's still too confusing. Too much terminology, too many settings. 3 pages and over half a dozen screenshots to explain how to make a bucket private. Too complicated. Google Drive folder permissions are easier. Phrases like "Anyone with the link can view" are understandable. Phrases like "Block public and cross-account access to buckets that have public policies" are not. Hide the fine-grained control in an "Advanced" panel, for those who really need it.
- ABeeSea 8y agoI’m pretty sure buckets are private by default.
- shepardrtc 8y agoThey are private by default. This is just making public/private settings very explicit, and also helping to ensure that anything that was public before the default setting was changed to private is known. This is really just extra CYA because people who really don't know what they are doing are apparently still being put in charge of buckets by companies.
- privateSFacct 8y agoTotally true! Some folks use S3 kind of a like a quick dropbox to share big files even internally in an org. Bob in accounting to Sue in finance. One solution I've found if folks INSIST they do not want any login / security. Expose the bucket contents using time expiring shared links (presigned links). This has worked great in one use case I had where users INSISTED on no security with sensitive data. Users can then cut / paste / download / share links for a few hours (all they really want - send it to a few folks on a project). Even if their email is hacked you are OK. Not ideal, but better than nothing. If you look inside orgs, the desire for no security is sometimes surprisingly high!
- isostatic 8y ago> the desire for no security is sometimes surprisingly high! The desire for security to not get in the way is high.
- privateSFacct 8y agoMy two wishes: Totally private (no permission) buckets by default. Some additional support for common usecases that drive the shortcut to public readable (often sharing internally for a project). Not sure if this would be a good thing, but folks are used to defining a group, adding users to a group (project / departments etc) in enterprise context, AWS supports this with IAM, but I wonder if surfacing a UI / something here might simplify this in some way.
- jeffbarr 8y agoPlease feel free to track me down and email those use cases to me.
- privateSFacct 8y agoHi Jeff, I'm not sure of the answer, but: Bob says I'd like to play around with data x on S3, or grab report z on S3. They are used to the Dropbox / drive / box etc model. That's often invite by email or allow *@domain.com permission. But how does this work in S3, how do you invite someone, how do they self provision their credentials without admin involvement etc? For a non aws/s3 user? One thing you can do, make bucket world readable. Big user base plus more stuff collecting in S3 from various systems seems to drive some risk, not sure how s3 solves for above or if it should (email or domain invite, self provisioning of credentials if not on platform etc). Tie into workdocs somehow? Push into competitor tools? This issue also comes up in the cross account / org situation. Want to invite someone outside of org to read data etc. Tools to do this, but it takes more work than folks are used to with other approaches. Pretty soon something is world readable. API side for developers works great. Be curious if others had use cases that create gravity towards world readable - because s3 by default is not, but a relatively high number of people end up there for some reason. Be interesting to try to give them what they want in addition to lockdown tools.
- dorianm 8y agoHmmm https://www.google.com/search?q=site%3As3.amazonaws.com&en= https://www.google.com/search?q=site%3As3.amazonaws.com&en= Combine it with the right keywords and you can find anything
- bdcravens 8y agoEverything I found was either a PDF file (common use case when you'd want the bucket to be public) or an html file in a bucket-hosted website (another use case where public is the correct behavior)
- tombrossman 8y agoRelated question, anyone know how to grant access to S3 objects via CloudFront only, since CloudFront bandwidth usage is cheaper than S3 objects accessed directly? I've got it working for top level documents using Origin Access Identity, but subfolders (example.com/test/index.html) doesn't work. I'm surprised this use case isn't better documented because it saves you money and you don't need to make your bucket public.
- mvanbaak 8y agohttps://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope... With a bucket policy you can use wildcards etc to provide access to all or parts of the bucket content.
- ReidZB 8y agoDepending on where your original bucket is hosted, it is very possible that CloudFront ends up costing more in data transfer. For instance, a us-east-1 bucket has a data transfer rate to the internet starting at $0.09/GB. That's flat, regardless of where the requester is located. On the other hand, while US- and Europe-based transfer pricing is cheaper in CloudFront (starting at $0.085/GB), all the other regions are more expensive... for example, South America starts at $0.25/GB (!). Of course the reverse is true: a bucket originally hosted in South America would probably do well to enable CloudFront, as requests from every other region become cheaper... And yet another wrinkle, I think CloudFront always is cheaper than S3 eventually (with enough data usage). So for very large customers, maybe it is true that CloudFront will always be cheaper. Clear as mud.
- morpheuskafka 8y agoIf people using AWS for your organization can't be bothered to take a few minutes to RTFM in its entirety (AWS has excellent documentation, compare to Azure's docs.microsoft.com behemoth of confusion), they shouldn't have access to reconfigure you're entire cloud infrastructure. Buckets are sensible private by default. If you don't know what you're doing, leave them that way.
- kderbyma 8y agoThis is sarcasm right? AWS doc's are okay, but not great. Honestly, their documents are pretty piss poor. They have contradicting statements depending on what tool, they have 100 exceptions for integrating their own toolchain, and the perspective and context is never clear. Sure...their basic stuff is well documented....but it's constantly changing and their system isn't versioned in a way that is easy to upgrade or transition legacy systems.
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]