6 ms·

Amazon S3 will now encrypt all new data with AES-256 by default

38 points by mikece 4y ago

joshstrange 4y ago
> Two notable examples concerning Amazon S3 storage buckets are the leak of data from 123 million households in December 2017 and the leak of 540 million records of Facebook users in April 2019. > If that data had been encrypted, the leaks wouldn't have had nearly as dire consequences for the exposed individuals, but unfortunately, due to overhead costs, operational complexity, and performance sacrifices, database encryption is commonly avoided. Ok, I consider myself semi-knowledgeable about these things and I use AWS for professional and personal projects but this confused me. Deciding to encrypt your S3 bucket only means encryption at rest right?. Of course you could encrypt your data before storing it in S3 but that's a different thing. This article seems to think that if encryption was on by default that public buckets wouldn't have leaked. If you set your bucket to public then AWS is going to decrypt your data on the fly when someone goes to ask for it right? Or am I missing something?
- ryanisnan 4y ago
  No, you are correct. The author seems to be confused about how EAR works.
- ctvo 4y ago
  No, I think you're right. The author conflates Amazon's own access to data encrypted at rest, which is what this appears to be about, with other security issues. This will help prevent, say, old hard drives being disposed of improperly, concerns around AWS employees easily viewing your data, etc.. Nothing will help if the credentials that grant access to read the encrypted data are what's compromised.
  - TillE 4y ago
    If decryption takes place on a different machine than the stored data, there's plenty you can do to separate how those are accessed. It's not a perfect system, but it is a legitimate extra layer of security.
    - ctvo 4y ago
      I'm unsure which point you're following up on. Yes, it's a legitimate extra layer of security if you do what you're suggesting, but that's not what's occurring here, though it may be what's happening within AWS's own data centers. But again, let's not conflate the two. From a customer's perspective: calling AWS will return the data unencrypted regardless of if this flag is on because AWS will decrypt before returning it, seamlessly. If the credentials allow access to the S3 bucket, that's that, and your data will still be compromised regardless if this feature is on or off.
- eigen-vector 4y ago
  [AWS engineer here] You're right. It's encryption at rest. The article reads like it's been written by someone who doesn't understand how those leaks occurred, what encryption means, or what has been launched here. I'd recommend changing from this article to the official announcement from AWS. https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-objects-by-default/ https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-obje...
  - steffanA 4y ago
    Actually, looks like they understand how the leaks occurred. Just not that AWS automatically decrypts the files, making it unable to protect against those types of breaches.
- fortituded0002 4y ago
  I don't think you are off the mark. (from https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingE...) > Server-Side Encryption – Request Amazon S3 to encrypt your object before saving it on disks in its data centers and then decrypt it when you download the objects. Yes it looks like it's at rest. It automatically decrypts if you go through S3 APIs to access the objects. The big leaks were due to S3 buckets being public which I believe means that this new encryption on by default won't actually help. The only thing Encryption at rest gives you is comfort in knowing if Amazon servers get compromised - not if your servers that have permissions to pull the data get compromised. With that said, it is a good step in the right direction. And additional security added helps address at least some use cases.
- belter 4y ago
  What you and others replying to you, seem to be missing, is that there are several flavors of Server Side Encryption, and where the control layer is concerning the access to the key when you ask for decryption. The article did not explain it well, and in any case the automatic option being enabled here, will not achieve this additional control level. If the leak was due to credentials leak would not help. But if the leak was because somebody accidentally made a server side object public AND you are using a KMS customer key then you have an additional layer of control required. You can easily see that, if you make a server side encrypted object on S3 public, but you are using a KMS customer key, you will get a different message from the default decryption of the object granted by this new default... You will get something like this <Error> <Code>InvalidArgument</Code> <Message> Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4. </Message> <ArgumentName>Authorization</ArgumentName> <ArgumentValue>null</ArgumentValue> <RequestId>B24FYZDV760J6TK0</RequestId> <HostId> Xi2fLZvfHxu5VopeLIrP292lg7WMV5ubVaBVpFdlynjL6V08IbdEFWYJuYwQZdLP/cZF2USWdNg= </HostId> </Error> This blog might help: "How to use KMS and IAM to enable independent security controls for encrypted data in S3" - https://aws.amazon.com/de/blogs/security/how-to-use-kms-and-iam-to-enable-independent-security-controls-for-encrypted-data-in-s3/ https://aws.amazon.com/de/blogs/security/how-to-use-kms-and-...
CommanderHux 4y ago
Surprised it took this long for S3 to enable this by default. Azure Storage did it back in October, 2017 for new objects and retroactively applied it for existing objects too as a background process. https://learn.microsoft.com/en-us/azure/storage/common/storage-service-encryption#about-azure-storage-service-side-encryption https://learn.microsoft.com/en-us/azure/storage/common/stora... Google Cloud storage also has it available by default, but not sure if this was always the case. https://cloud.google.com/storage/docs/encryption https://cloud.google.com/storage/docs/encryption
- jorams 4y ago
  It seems Google Cloud Storage introduced it in August 2013[1], enabled by default. [1]: https://cloudplatform.googleblog.com/2013/08/google-cloud-storage-now-provides.html https://cloudplatform.googleblog.com/2013/08/google-cloud-st...
spullara 4y ago
Encryption at rest (where the customer isn't the sole owner of the key) applies to a very dumb attack vector, disk drives found in a landfill that somehow weren't wiped and shredded as they left the DC.
- wereallterrrist 4y ago
  On the one hand, fair, I hadn't considered that eventuality. Still, I can't imagine a scenario where, "I need to be able to guard against [cloud provider] blobs being left unencrypted for the long-term" that doesn't also include "and protect against two of their internal services being co-compromised". If I can't trust blobs in a landfill, I wouldn't trust blobs in a landfill that are decrypted by another blob in the same landfill. Just feels like smoke and mirrors that is purely about chasing CTOs and "CYA"-type decisions.
- ericbarrett 4y ago
  It's not usually landfills which are the vector for this kind of leak, it's second-hand disk sales. This includes both decommed hardware and items scavenged from failures: often the failure was transient, or in the chassis; the maintenance vendor will test and resell the disk if it passes. They are supposed to be wiped, but this takes a long time and is almost never verified so the process is often incomplete. Some such vendors have a blanket destruction policy for old hardware, but this obviously means higher prices, so it's usually only borne by medical and military institutions. And there have been notable cases where this was supposed to happen but the disks ended up on eBay, unwiped, regardless. SSDs make this even more likely because they have pools of storage which are screened off from normal access but are typically readable through special code in the firmware; as failing sectors are swapped out, sensitive plaintext can remain behind, unable to be wiped by a conventional dd-style blast. (HDDs have this too, but in much smaller quantities.)
  - spullara 4y ago
    I don't believe that AWS sells its retired equipment, at least I could find no evidence for it and thus I think the likelyhood that someone gets a hold of an unwiped/unshredded SSD from them to be exceedingly low. I am surprised that drive manufacturers haven't built in encryption at rest though since it would be pretty easy to generate a key on first boot and then make it easy to wipe by just making a new key and thus essentially scrambling all the data on the disk instantly without having to wipe them at all.
orf 4y ago
This article is pretty rubbish: 1. S3 data events cost money and cannot be enabled “at no extra charge” 2. This wouldn’t have prevented the referenced leaks 3. The link to “how to re-encrypt objects on s3” goes to a horribly outdated and over complex solution. Use s3 batch operations. This is a better link: https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-objects-by-default/ https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-obje...

38 points by mikece 4y ago

> Two notable examples concerning Amazon S3 storage buckets are the leak of data from 123 million households in December 2017 and the leak of 540 million records of Facebook users in April 2019. > If that data had been encrypted, the leaks wouldn't have had nearly as dire consequences for the exposed individuals, but unfortunately, due to overhead costs, operational complexity, and performance sacrifices, database encryption is commonly avoided. Ok, I consider myself semi-knowledgeable about these things and I use AWS for professional and personal projects but this confused me. Deciding to encrypt your S3 bucket only means encryption at rest right?. Of course you could encrypt your data before storing it in S3 but that's a different thing. This article seems to think that if encryption was on by default that public buckets wouldn't have leaked. If you set your bucket to public then AWS is going to decrypt your data on the fly when someone goes to ask for it right? Or am I missing something?

ryanisnan 4y ago
No, you are correct. The author seems to be confused about how EAR works.
ctvo 4y ago
No, I think you're right. The author conflates Amazon's own access to data encrypted at rest, which is what this appears to be about, with other security issues. This will help prevent, say, old hard drives being disposed of improperly, concerns around AWS employees easily viewing your data, etc.. Nothing will help if the credentials that grant access to read the encrypted data are what's compromised.
- TillE 4y ago
  If decryption takes place on a different machine than the stored data, there's plenty you can do to separate how those are accessed. It's not a perfect system, but it is a legitimate extra layer of security.
  - ctvo 4y ago
    I'm unsure which point you're following up on. Yes, it's a legitimate extra layer of security if you do what you're suggesting, but that's not what's occurring here, though it may be what's happening within AWS's own data centers. But again, let's not conflate the two. From a customer's perspective: calling AWS will return the data unencrypted regardless of if this flag is on because AWS will decrypt before returning it, seamlessly. If the credentials allow access to the S3 bucket, that's that, and your data will still be compromised regardless if this feature is on or off.
eigen-vector 4y ago
[AWS engineer here] You're right. It's encryption at rest. The article reads like it's been written by someone who doesn't understand how those leaks occurred, what encryption means, or what has been launched here. I'd recommend changing from this article to the official announcement from AWS. https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-objects-by-default/ https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-obje...
- steffanA 4y ago
  Actually, looks like they understand how the leaks occurred. Just not that AWS automatically decrypts the files, making it unable to protect against those types of breaches.
fortituded0002 4y ago
I don't think you are off the mark. (from https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingE...) > Server-Side Encryption – Request Amazon S3 to encrypt your object before saving it on disks in its data centers and then decrypt it when you download the objects. Yes it looks like it's at rest. It automatically decrypts if you go through S3 APIs to access the objects. The big leaks were due to S3 buckets being public which I believe means that this new encryption on by default won't actually help. The only thing Encryption at rest gives you is comfort in knowing if Amazon servers get compromised - not if your servers that have permissions to pull the data get compromised. With that said, it is a good step in the right direction. And additional security added helps address at least some use cases.
belter 4y ago
What you and others replying to you, seem to be missing, is that there are several flavors of Server Side Encryption, and where the control layer is concerning the access to the key when you ask for decryption. The article did not explain it well, and in any case the automatic option being enabled here, will not achieve this additional control level. If the leak was due to credentials leak would not help. But if the leak was because somebody accidentally made a server side object public AND you are using a KMS customer key then you have an additional layer of control required. You can easily see that, if you make a server side encrypted object on S3 public, but you are using a KMS customer key, you will get a different message from the default decryption of the object granted by this new default... You will get something like this <Error> <Code>InvalidArgument</Code> <Message> Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4. </Message> <ArgumentName>Authorization</ArgumentName> <ArgumentValue>null</ArgumentValue> <RequestId>B24FYZDV760J6TK0</RequestId> <HostId> Xi2fLZvfHxu5VopeLIrP292lg7WMV5ubVaBVpFdlynjL6V08IbdEFWYJuYwQZdLP/cZF2USWdNg= </HostId> </Error> This blog might help: "How to use KMS and IAM to enable independent security controls for encrypted data in S3" - https://aws.amazon.com/de/blogs/security/how-to-use-kms-and-iam-to-enable-independent-security-controls-for-encrypted-data-in-s3/ https://aws.amazon.com/de/blogs/security/how-to-use-kms-and-...

CommanderHux 4y ago

Surprised it took this long for S3 to enable this by default. Azure Storage did it back in October, 2017 for new objects and retroactively applied it for existing objects too as a background process. https://learn.microsoft.com/en-us/azure/storage/common/storage-service-encryption#about-azure-storage-service-side-encryption https://learn.microsoft.com/en-us/azure/storage/common/stora... Google Cloud storage also has it available by default, but not sure if this was always the case. https://cloud.google.com/storage/docs/encryption https://cloud.google.com/storage/docs/encryption

jorams 4y ago
It seems Google Cloud Storage introduced it in August 2013[1], enabled by default. [1]: https://cloudplatform.googleblog.com/2013/08/google-cloud-storage-now-provides.html https://cloudplatform.googleblog.com/2013/08/google-cloud-st...

spullara 4y ago

Encryption at rest (where the customer isn't the sole owner of the key) applies to a very dumb attack vector, disk drives found in a landfill that somehow weren't wiped and shredded as they left the DC.

wereallterrrist 4y ago
On the one hand, fair, I hadn't considered that eventuality. Still, I can't imagine a scenario where, "I need to be able to guard against [cloud provider] blobs being left unencrypted for the long-term" that doesn't also include "and protect against two of their internal services being co-compromised". If I can't trust blobs in a landfill, I wouldn't trust blobs in a landfill that are decrypted by another blob in the same landfill. Just feels like smoke and mirrors that is purely about chasing CTOs and "CYA"-type decisions.
ericbarrett 4y ago
It's not usually landfills which are the vector for this kind of leak, it's second-hand disk sales. This includes both decommed hardware and items scavenged from failures: often the failure was transient, or in the chassis; the maintenance vendor will test and resell the disk if it passes. They are supposed to be wiped, but this takes a long time and is almost never verified so the process is often incomplete. Some such vendors have a blanket destruction policy for old hardware, but this obviously means higher prices, so it's usually only borne by medical and military institutions. And there have been notable cases where this was supposed to happen but the disks ended up on eBay, unwiped, regardless. SSDs make this even more likely because they have pools of storage which are screened off from normal access but are typically readable through special code in the firmware; as failing sectors are swapped out, sensitive plaintext can remain behind, unable to be wiped by a conventional dd-style blast. (HDDs have this too, but in much smaller quantities.)
- spullara 4y ago
  I don't believe that AWS sells its retired equipment, at least I could find no evidence for it and thus I think the likelyhood that someone gets a hold of an unwiped/unshredded SSD from them to be exceedingly low. I am surprised that drive manufacturers haven't built in encryption at rest though since it would be pretty easy to generate a key on first boot and then make it easy to wipe by just making a new key and thus essentially scrambling all the data on the disk instantly without having to wipe them at all.

orf 4y ago

This article is pretty rubbish: 1. S3 data events cost money and cannot be enabled “at no extra charge” 2. This wouldn’t have prevented the referenced leaks 3. The link to “how to re-encrypt objects on s3” goes to a horribly outdated and over complex solution. Use s3 batch operations. This is a better link: https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-objects-by-default/ https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-obje...