8 ms·
The Cloud Conundrum: S3 Encryption
- daitya 4y agoAWS will now encrypt all new data in its Amazon S3 storage service by default. Huge announcement, secure default for the win, sure, but it gives a false sense of security.
- tgv 4y agoAFAIK, it only helps against hackers/insiders who steal the raw file. If you provide access to your bucket, it isn't doing anything. As always with Amazon (and other services, really): you really should understand the service. Just using one because you read "encryption" is unwise.
- limitedsupply 4y agoWouldn’t your example apply to any encrypted hard drive as well? If you provide access to it encryption is not doing anything.
- NathanKP 4y agoIt's all about layers of protection. This layer of server side encryption is designed to protect against an attack where someone breaks into an AWS data center, pulls a disk drive out of a storage system that is hosting S3 objects, and then tries to read the data off of that disk drive. Without the key (which is obviously stored separately, on a separate system) the disk will be useless to them.
- ilyt 4y agoWhich is like the least likely attack to happen.
- lokar 4y agoI surprised they did not always encrypt data at rest, google does.
- advisedwang 4y agoSSE-S3/SSE-KMS does provide defense against a few things: * Lost/stolen hard disks from AWS datacenter * Attacker or rogue AWS employee who has access to low level storage but not keys. The article is correct that this misses many important situations but it is still better than nothing.
- wrldos 4y agoThese are far less likely outcomes than your root account being compromised, an IAM fuck up, dumb assume role configuration or something using an assumed IAM role being compromised and having verbatim access to the bucket. The feature is there for paperwork reasons.
- TavsiE9s 4y agoIt's there so folks can check their list and go "Yep, it's encrypted at rest. Yay us!"
- thallium205 4y agoSOC2 here we come!
- daitya 4y agoAuthor here. Exactly, that's the biggest risk!
- oxfordmale 4y agoIf your root account is comprised, it is game over anyway.
- maximilianroos 4y ago"less likely" risks are still worth protecting against! I still put my seat-belt on, even though there's an airbag in the car.
- klodolph 4y agoLess likely? Yes. However, these are real, legitimate risks that need to be mitigated! This is not just some faff that needs to get put on paper. Think about your risk model: there are a lot of employees at your cloud provider, and a lot of high-profile customers. There's a defense in depth approach here. Metal detectors, security guards, and searches to enter & exit data centers. Data encrypted when on-disk. Keep the keys in some KMS system. Physically isolate the KMS system, use HSMs, restrict permissions to the KMS. Without these measures, you might end up with a system where ordinary customer support representatives, software engineers, data center technicians, or system administrators / SREs can exfiltrate customer data on a whim. At a large cloud provider like Amazon, Google, or Microsoft, that's not a small attack surface. That's a lot of people. The idea is to make it extremely hard for one of these employees to exfiltrate customer data without somehow leaving a record of it which ultimately gets the employee fired and prosecuted. Customers will make mistakes too--screw up their permissions, fail to encrypt their data, or whatever. The idea behind defense in depth is that any one security measure can probably be circumvented, but multiple independent security measures are harder to circumvent. It's really easy to be cynical about security but I have worked at cloud providers and the threats are real.
- advisedwang 4y agoI wonder if the real reason for encrypting is that it allows cheaper deletion. You can save the IO of deleting an entire file and just delete the encryption key. (aka cryptographic erasure, see [1]) [1] https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-88r1.pdf https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.S...
- throwaway320498 4y agoNot this. A customer's bytes are likely spread across millions of hard drives. At the filesystem layer, the region of the disk those bytes occupied still need to be reclaimed. What you're describing makes sense for a service like EBS where a single customer's bytes are limited to small number of disks in well-defined locations.
- hdjjhhvvhga 4y agoOne problem with utilizing unencrypted drives is that if there's a hardware failure, it's more expensive to dispose of them. Normally Amazon overwrites the data, degausses the drive and then destroys it, but if, say, the electronics is faulty, the first step becomes more difficult.
- gregw2 4y agoMy speculation is that they now have nitro encryption on all or enough of the servers supporting s3 so it’s now no marginal cost for them anymore since it’s inline ASIC encryption and not taking incremental CPU cycles anymore.
- severino 4y agoThis also implies that if the encryption key is that easy to dispose of, then it's also easy to somehow lose or screw it turning your valuable client data into a random stream of bytes, doesn't it?
- advisedwang 4y agoThey can dispose of it in the same way that they dispose of the actual core data (ie write over it X times, and ensure when disk is EOL that it's shredded). With cryptographic erasure you can do exactly the same erasure process on fewer bytes and get the same effective results.
- deleted 4y ago[deleted]
- davewritescode 4y agoWhat’s always been unclear to me is how much I actually gain from providing my own KMS key use with S3 if at the end of the day my private key is available to the s3 service. I would only use client side encryption/decryption if I had really sensitive data and I truly couldn’t trust AWS with it.
- paranoidrobot 4y agoAssuming for a moment that AWS's security at all matches the public API. There are two ways to grant access to a KMS key: as a policy grant on a user/role, and as a policy on the KMS key itself. If you don't have access to the key, then reading the S3 object doesn't work. So I have to trust that AWS's security here is that the S3 API itself can't unless granted that access.
- theteapot 4y ago> If you don't have access to the key, then reading the S3 object doesn't work. I'm still not convinced. Just seem like theater. S3 has its own access control. If you don't have access to the S3 object then reading the S3 object doesn't work ... Requiring additional access to the key seems redundant - theater. P.S. I'm not arguing against encryption at rest. I'll take it, but using my own key doesn't seem to offer too much over using one of AWS's rel low level access.
- colmmacc 4y agoAWS isn't monolithic internally and we use permissions boundaries between systems like S3 and KMS (in fact they are mediated by IAM). There's some more detail in my talk from re:Invent - https://www.youtube.com/watch?v=kNbNWxVQP4w https://www.youtube.com/watch?v=kNbNWxVQP4w . For a case like this, the upshot of it is that with permissions on the KMS key, S3 is effectively locked out and can't fetch it. KMS is a hermetic system with no operator access with HSMs at its root, so that provides a pretty meaningful difference. It also means that controls can be handled by different customer teams, and that a single change on a shared CMK can render an unbounded number of S3 objects inaccessible quickly.
- foota 4y agoI was amazed when I read the other week it wasn't already encrypted at rest by default.
- icedchai 4y agoYou could enable it on the bucket level for many years now. This would apply to all new objects in the bucket. So if you enabled it at bucket creation time, it was effectively "by default."
- frenchman99 4y agoPeople here say that this doesn't secure anything. But in addition to encryption by default, AWS announced they will as of April 2023 enforce secure S3 configuration (no public access) by default. So things are definitely getting better when you put these announcements together. https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/ https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security...
- jbverschoor 4y agoWell it’s kind of insane that it has been open by default, resulting in SOOOO many leaks. They might as well have open SSH access on all EC2 instances using root + “password”
- beaviskhan 4y agoThat's not entirely fair - the default has been to ALLOW people to set buckets to public and/or to set public ACLs on individual objects. Both of these require the customer to Do Something in order to grant public access. The default has NOT been "public access for all" ever, that I am aware of.
- fdr 4y agoThis definitely dates back to when offloading images and other static assets to S3 from your (era-appropriate) Apache or Varnish server was an optimization of the day...now, obsolete, as, at the time, there were not low-friction, lower-cost CDNs available, such as Cloudfront or Cloudflare.
- neilv 4y agoEarlier in AWS, when I was migrating a complex sensitive system from self-hosted servers, there was a requirement not to leave data at rest in S3 unencrypted, and there was no suitable S3 feature for this. Since I had to write the AWS API from scratch anyway (fringe language, and we tried to avoid C memory problems), I also implemented transparent encryption at the same time. (Using an off-the-shelf vetted implementation of appropriate encryption algorithms. And in such a way that, when handling large data, it could run on other cores if no accelerator.) Of course, as the article points out, the EC2 instances (or whatever needs to access the data within the objects in S3) by definition need access, but at least that access is restricted more closely to what actually needs it.
- DethNinja 4y ago“Users” in this case are supposed to be software engineers. Thus, I don’t see how Amazon’s claim can be misleading. Server side encryption is very valuable, especially against breaches on the cloud provider. I would expect any mid to senior level engineer to understand the implications and add further security (local encryption before upload etc.) measures as neeeded.
- ilyt 4y agoBreaches in cloud provider so far tend to be entirely software, at least for the big ones. Encryption at rest doesn't help prevent software bugs
- specialp 4y agoThere is a danger too to encrypting your data with non built in KMS key: The key can be deleted and your data can become irrecoverable. KMS has a waiting period of 7-30 days when it is deleted but it is up to you to alert yourself of the fact that someone deleted it. We have AWS config alerts on this that send out via various means.
- wiredfool 4y agoEncryption is easy. Key management is hard.
- aborsy 4y agoAWS has access to the AWS-managed SSE-KMS keys, since AWS manages those keys on behalf of customers. I am wondering if AWS has access to data encrypted with the SSE-CMK keys as well? They state that the SSE-CMK keys reside in tamper-proof FIPS-validated HSMs and not even Amazon has access to them. But it’s a bit misleading because they use envelope encryption. Furthermore, the encryption and decryption are still server side. They could have better phrased it that, the only advantage of the SSE-CMK keys over SSE-KMS keys is that, with former, the users can define key policies (providing another access control layer similar to IAM) and monitor usage.
- daitya 4y agoAuthor here. Technically, they can. HOWEVER if this threat is part of a customer's risk profile, i.e., they cannot risk even a possibility of AWS having access to their data, then use client side encryption.
- zxcvbn4038 4y agoI believe all of the AWS S3 SDKs except php support client side encryption, you could also roll your own. In my mind that is more secure then S3 SSE or KMS because you are the only one with the key.
- aborsy 4y agoThere is huge amount of data in S3 and EC2’s. Other cloud providers such as GCP hold similarly large amounts of data. The governments are no doubt very interested in these gold mines. I am curious if cloud providers such as AWS provide default access to data stored in their data centers to the governments (to all data, and by default, in a successor to the PRISM program, not on a case by case basis and under subpoenas which is a different program)? If so, KMS doesn’t help, and client-side encryption is the only way to protect the data.
- belter 4y agoOr a CloudHSM if you trust the certification: https://aws.amazon.com/cloudhsm/ https://aws.amazon.com/cloudhsm/
- 0xbadcafebee 4y ago> Next, at best SSE-S3 adds a defense in depth protection against a physical loss, theft or confiscation of an AWS hard drive storing your data. Think crazy scenarios like a tornado or fire, followed by more chaos and somehow the AWS hard drive landing at Goodwill. If the data on it is unencrypted, game over. As you can imagine, the likelihood of this happening is about the same as that of the United States winning a cricket world cup. It's incredibly easy to dumpster dive old hardware from datacenters (or waste management facilities). Even if their policy is to demagnetize and shred every hard-drive out of every dead machine, that's assuming they always follow their policy correctly. Humans are flawed (and subject to bribes). I have friends who run K8s clusters on hardware they took from the dumpsters behind datacenters. (Not Amazon datacenters, but ones where the government has an entire floor dedicated to them)