11 ms·

Amazon DynamoDB encrypts all customer data at rest

91 points by petercooper 8y ago

deleted 8y ago
[deleted]
gravypod 8y ago
In AWS' HIPAA compliance docs they recommend not relying on their managed encryption services as they currently protect you from a legal standpoint but may not in the future (at least for RDS). Is AWS altering those guidelines and now going to say their encryption is good enough?
- joelhaasnoot 8y ago
  Isn't this true in general? This protects against someone walking into the datacenter and stealing a harddrive (which is Amazon's responsibility anyway). If you want to be sure the data is encrypted, store encrypted values/blobs/files/etc is generally Amazon's stance (for instance, for S3). For S3 they call this "client side encryption".
ejcx 8y ago
So much cloud encryption is a total checkbox. As a security professional I sometimes don't have a clear threat model that the magic crypto is meant for.
- twimbley 8y ago
  The only threat in the room that makes sense is the rogue, disgruntled amazon employee. One might hypothesize government auditors and warrants compelling raw disk contents to be turned over for criminal prosecution, but that scenario supposes that a government entity would lack the creativity to imagine options like arresting the root login user, and jailing them until they enter the root account password to offer access to the encrypted data at rest.
  - kefabean 8y ago
    True and even this assumes a specific limited threat vector: eg. employee has access to storage volume but not encryption key, employee does not have ability to dump guest memory, employee has no backdoor mechanism to elevate permissions within account. We have no cast iron guarantees (other than trust) that these limiting controls are in place.
    - jiveturkey 8y ago
      Sure, but those things require “lots” of access. Just grabbing a disk or decommissioning half of an HA pair etc requires only “little” access. Just because there are complex vectors doesn’t mean you give up on the basic ones. This is a valuable thing, and for now, a differentiator.
- mongro1 8y ago
  The problem for us is the mystery of everything that underpins the service. For managers, auditors, regulators, the tickbox allows us to get on with our jobs.
  - jiveturkey 8y ago
    Thank you. I’m amazed so many here don’t get it. How are you guys building robust things? Maybe I’m too far along to see from the eyes of a novice, but why would anyone poo-poo default encryption at rest?! And as-a-service to boot!
- jnwatson 8y ago
  Yeah i’m confused as to the threat model. It isn’t like they ship hard drives around. It is like taping the key above the door lock. Access control to the door is the important part.
  - azurezyq 8y ago
    With encryption at rest, you can basically make sure data can only be accessed from the main interface. Otherwise there are multiple ways to let data leak, e.g., sys admins, decommissioned disks, or other tools.
    - ec109685 8y ago
      Sys admins would have access regardless if they were evil enough.
      - smueller1234 8y ago
        It's entirely feasible to mitigate a lot of that insider risk. You have to pick the convenience/risk reduction trade off, though. For example, it may make sense to disallow any unilateral access, and only allow signed, reviewed code to run. There doesn't have to be a single entity with universal root. In such an environment, encryption at rest can come in handy because it mitigates some of the physical attack vectors.
        ec109685 8y ago
        It definitely helps for physical attack vectors.
    - finchisko 8y ago
      In theory yes. In practice most of the tasks including key/password management is left on sysadmins anyway, because management don't bother and not even interested how this stuff works. They're just not technical people, their priorities are elsewhere. And that's ok.
  - 8y ago
- eksemplar 8y ago
  The European public sector is moving into the cloud and we’re picking either Amazon or Azure. Azure is really far ahead on security compliance with EU regulation, perhaps mostly on the image side of things though. But that’s important too, because when I ask our data protection officer and our lawyers which way to go, they point to Azure. Which is great and bad. I really, really like Microsoft and Azure is awesome for .Net. Only we’re doing less and less .Net, and more and more of the development pool I have to chose from in hiring situations live in AWS. Our main cloud will continue to run in Azure, because that’s where our 365 platform is, it’s where our IT crew is certified and Microsoft has been a good partner for decades. But I do want the option for us to use smaller software houses that are AWS only to develop minor systems that we operate in AWS ourselves as well. I think this will help AWS in that regard.
  - touristtam 8y ago
    It's a bit of a concern that the only alternative Europeans really have are all american at a time where those are seen not so much as friendly by the public at large but the lesser of two evil when compared with what Alibaba is offering.
    - easytiger 8y ago
      Almost everything they are doing could be lifted and shifted to generic VPS hosts, but no... everything has to be "cloud based", meaning they have to be rearchitected to accomodate the fragility and inherent flaws of that paradigm; namely that you have no control over your physical comms or who is accessing your hosts.
      - foolfoolz 8y ago
        at large enough scale, VPS hosts are just as fragile. physical comms or access is really about the same on a cloud host as anywhere else. good luck getting something like IAM on a VPS provider. you can totally run off a VPS. as long as you have a server reliability team as good as a cloud service provider. and a management console as good as a cloud vm manager. and a way to grant access, and monitor, and grow on demand, and etc.
- tylerl 8y ago
  Encrypted storage protects a layer from every layer below it. It simplifies your defense model. It's pretty much always a good idea to do, but rarely something worth bragging about.
- londons_explore 8y ago
  The threat model is "someone broke into Amazon's data center and stole the hard drives". That's a silly threat model, but it takes a long time for rules and regulations to catch up with realities.
wanghq 8y ago
How to do range query if the sort/range keys are encrypted? https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...
- deleted 8y ago
  [deleted]
- guitarbill 8y ago
  well, the term is at rest. so an index in memory is probably not encrypted, just as they decrypt the data to be send over the wire (also encrypted, e.g. via HTTPS, which I think the SDK uses under the hood). this isn't a big deal since sensitive/user data usually is a terrible choice for a key. (you get hot spots.) but i'm sure there's someone out there using SSNs as a key...
winrid 8y ago
So how do they do aggregations then?
- zip1234 8y ago
  Not encrypted in memory, just on disk.
- erik_seaberg 8y ago
  They don't; DynamoDB is a partitioned document store. You can query for a range of a sorted key but it just gives you the documents in that range. And I'm a little fuzzy on KMS but I believe the entire table is encrypted with one key, so if they could do SUM(x) GROUP BY y if they wanted to take the scaling hit.
raghava 8y ago
I might be missing something, but wasn't this (=> encryption at rest) the case always, and why this post only now? Also, the cautionary note, as given https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.howitworks.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... "You should ensure you have configured the SDK to reuse connections. Otherwise you will experience latencies from DynamoDB having to re-establish new KMS cache entries for each DynamoDB operation, and potentially have to face higher KMS and Cloudtrail costs. For example, to do this using the Node.js SDK, you can create a new https agent with keepAlive turned on. "
- nostrebored 8y ago
  That data is encrypted in-flight
  - jiveturkey 8y ago
    we call that in-transit. at-rest, in-transit, in-use
redwood 8y ago
Odd this only happened now...
romed 8y ago
I guess I assumed all data at AWS was encrypted at rest. Disturbing if not. Compare: https://cloud.google.com/security/encryption-at-rest/default-encryption/ https://cloud.google.com/security/encryption-at-rest/default...

91 points by petercooper 8y ago

deleted 8y ago

[deleted]

In AWS' HIPAA compliance docs they recommend not relying on their managed encryption services as they currently protect you from a legal standpoint but may not in the future (at least for RDS). Is AWS altering those guidelines and now going to say their encryption is good enough?

joelhaasnoot 8y ago
Isn't this true in general? This protects against someone walking into the datacenter and stealing a harddrive (which is Amazon's responsibility anyway). If you want to be sure the data is encrypted, store encrypted values/blobs/files/etc is generally Amazon's stance (for instance, for S3). For S3 they call this "client side encryption".

ejcx 8y ago

So much cloud encryption is a total checkbox. As a security professional I sometimes don't have a clear threat model that the magic crypto is meant for.

twimbley 8y ago
The only threat in the room that makes sense is the rogue, disgruntled amazon employee. One might hypothesize government auditors and warrants compelling raw disk contents to be turned over for criminal prosecution, but that scenario supposes that a government entity would lack the creativity to imagine options like arresting the root login user, and jailing them until they enter the root account password to offer access to the encrypted data at rest.
- kefabean 8y ago
  True and even this assumes a specific limited threat vector: eg. employee has access to storage volume but not encryption key, employee does not have ability to dump guest memory, employee has no backdoor mechanism to elevate permissions within account. We have no cast iron guarantees (other than trust) that these limiting controls are in place.
  - jiveturkey 8y ago
    Sure, but those things require “lots” of access. Just grabbing a disk or decommissioning half of an HA pair etc requires only “little” access. Just because there are complex vectors doesn’t mean you give up on the basic ones. This is a valuable thing, and for now, a differentiator.
mongro1 8y ago
The problem for us is the mystery of everything that underpins the service. For managers, auditors, regulators, the tickbox allows us to get on with our jobs.
- jiveturkey 8y ago
  Thank you. I’m amazed so many here don’t get it. How are you guys building robust things? Maybe I’m too far along to see from the eyes of a novice, but why would anyone poo-poo default encryption at rest?! And as-a-service to boot!
jnwatson 8y ago
Yeah i’m confused as to the threat model. It isn’t like they ship hard drives around. It is like taping the key above the door lock. Access control to the door is the important part.
- azurezyq 8y ago
  With encryption at rest, you can basically make sure data can only be accessed from the main interface. Otherwise there are multiple ways to let data leak, e.g., sys admins, decommissioned disks, or other tools.
  - ec109685 8y ago
    Sys admins would have access regardless if they were evil enough.
    - smueller1234 8y ago
      It's entirely feasible to mitigate a lot of that insider risk. You have to pick the convenience/risk reduction trade off, though. For example, it may make sense to disallow any unilateral access, and only allow signed, reviewed code to run. There doesn't have to be a single entity with universal root. In such an environment, encryption at rest can come in handy because it mitigates some of the physical attack vectors.
      - ec109685 8y ago
        It definitely helps for physical attack vectors.
  - finchisko 8y ago
    In theory yes. In practice most of the tasks including key/password management is left on sysadmins anyway, because management don't bother and not even interested how this stuff works. They're just not technical people, their priorities are elsewhere. And that's ok.
- 8y ago
eksemplar 8y ago
The European public sector is moving into the cloud and we’re picking either Amazon or Azure. Azure is really far ahead on security compliance with EU regulation, perhaps mostly on the image side of things though. But that’s important too, because when I ask our data protection officer and our lawyers which way to go, they point to Azure. Which is great and bad. I really, really like Microsoft and Azure is awesome for .Net. Only we’re doing less and less .Net, and more and more of the development pool I have to chose from in hiring situations live in AWS. Our main cloud will continue to run in Azure, because that’s where our 365 platform is, it’s where our IT crew is certified and Microsoft has been a good partner for decades. But I do want the option for us to use smaller software houses that are AWS only to develop minor systems that we operate in AWS ourselves as well. I think this will help AWS in that regard.
- touristtam 8y ago
  It's a bit of a concern that the only alternative Europeans really have are all american at a time where those are seen not so much as friendly by the public at large but the lesser of two evil when compared with what Alibaba is offering.
  - easytiger 8y ago
    Almost everything they are doing could be lifted and shifted to generic VPS hosts, but no... everything has to be "cloud based", meaning they have to be rearchitected to accomodate the fragility and inherent flaws of that paradigm; namely that you have no control over your physical comms or who is accessing your hosts.
    - foolfoolz 8y ago
      at large enough scale, VPS hosts are just as fragile. physical comms or access is really about the same on a cloud host as anywhere else. good luck getting something like IAM on a VPS provider. you can totally run off a VPS. as long as you have a server reliability team as good as a cloud service provider. and a management console as good as a cloud vm manager. and a way to grant access, and monitor, and grow on demand, and etc.
tylerl 8y ago
Encrypted storage protects a layer from every layer below it. It simplifies your defense model. It's pretty much always a good idea to do, but rarely something worth bragging about.
londons_explore 8y ago
The threat model is "someone broke into Amazon's data center and stole the hard drives". That's a silly threat model, but it takes a long time for rules and regulations to catch up with realities.

wanghq 8y ago

How to do range query if the sort/range keys are encrypted? https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...

deleted 8y ago
[deleted]
guitarbill 8y ago
well, the term is at rest. so an index in memory is probably not encrypted, just as they decrypt the data to be send over the wire (also encrypted, e.g. via HTTPS, which I think the SDK uses under the hood). this isn't a big deal since sensitive/user data usually is a terrible choice for a key. (you get hot spots.) but i'm sure there's someone out there using SSNs as a key...

winrid 8y ago

So how do they do aggregations then?

zip1234 8y ago
Not encrypted in memory, just on disk.
erik_seaberg 8y ago
They don't; DynamoDB is a partitioned document store. You can query for a range of a sorted key but it just gives you the documents in that range. And I'm a little fuzzy on KMS but I believe the entire table is encrypted with one key, so if they could do SUM(x) GROUP BY y if they wanted to take the scaling hit.

raghava 8y ago

I might be missing something, but wasn't this (=> encryption at rest) the case always, and why this post only now? Also, the cautionary note, as given https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.howitworks.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... "You should ensure you have configured the SDK to reuse connections. Otherwise you will experience latencies from DynamoDB having to re-establish new KMS cache entries for each DynamoDB operation, and potentially have to face higher KMS and Cloudtrail costs. For example, to do this using the Node.js SDK, you can create a new https agent with keepAlive turned on. "

nostrebored 8y ago
That data is encrypted in-flight
- jiveturkey 8y ago
  we call that in-transit. at-rest, in-transit, in-use

redwood 8y ago

Odd this only happened now...

romed 8y ago

I guess I assumed all data at AWS was encrypted at rest. Disturbing if not. Compare: https://cloud.google.com/security/encryption-at-rest/default-encryption/ https://cloud.google.com/security/encryption-at-rest/default...