31 ms·

Encryption at Rest: Whose Threat Model Is It Anyway?

199 points by chillax 2y ago

29athrowaway 2y ago
Encryption at rest is misleading. It is often used to describe an insecure situation consisting of disk encryption only instead of encrypted data store values.
- thom 2y ago
  The article explains why the former is often just security theatre.
  - tasn 2y ago
    Calling it security theater is a bit harsh, though it's definitely not a cure all solution. It does protect against physically taking of and improper disposal of HDs when done correctly. Which can be quite a significant vector.
    - klabb3 2y ago
      It’s not security theatre in the rare case that the stated goals is to protect against physical attacks, or for hardware disposal. If the goal is to put “military grade encryption” sticker in the sales deck, or to pass some certification, or to create a vague cloud of plausibility that you’re taking security seriously in some other way, then absolutely it’s security theatre. It deserves to be called out.
      - jeffbee 2y ago
        It's not "theater" if the customer requires that certification.
        mbb70 2y ago
        If the customer wants a show, give them a show. Doesn't make it not theater.
- doubled112 2y ago
  Encryption at rest is great if I forget my laptop on a bus. Reduces it to a VISA problem. But who is stealing data off of servers by taking the server? Maybe it saves you when you dispose of the disks? Maybe a home server in a break in? This has always seemed obvious to me.
  - zsims 2y ago
    There are other paths to the attack he mentioned. Eg you find an API that accepts ciphertext or part of. Or a cloud backup/restore flow. Likely you need another vulnerability but it does happen.
  - erinnh 2y ago
    I can think of two reasons: - Physically moving servers outside of the cages they are supposed to be in. (inside/outside DC) - Easier to dispose of disks (repair/EOL)
  - dspillett 2y ago
    > But who is stealing data off of servers by taking the server? Away from pure bare metal, in the presence of bugs in security separation someone could steal data simply by having a VM on the same server as yours. Encryption at-rest with each VM or service/account having their own keys removes the risk of data accidentally becoming visible in another environment that is sharing the hardware. Though the chance of some useful data being revealed this way is pretty small on large scale cloud installations, and if you are using a cheap VPS host for sensitive data you might not be valuing your data enough.
  - immibis 2y ago
    It has happened before that disks have been taken from servers in third-party data centers. Most commonly by the feds, but also by other actors. Do you control the physical security of your server?
candiddevmike 2y ago
This blog post is a lot of words to say encryption != authentication.
- photochemsyn 2y ago
  I think it's very valuable for people who might not already now this to see an example of why it's true, as the post does: > "What’s happening here is simple: The web application has the ability to decrypt different records encrypted with different keys. If you pass records that were encrypted for Alice to the application to decrypt it for Bob, and you’re not authenticating your access patterns, Bob can read Alice’s data by performing this attack." An interesting question here is whether or not there's a god key that allows the administrator to decrypt all the data even if they can't authenticate as the user (or if they just have copies of all keys). Searching HN for 'lavabits' turns up some results related to this, e.g. https://www.eff.org/press/releases/eff-has-lavabits-back-contempt-court-appeal https://www.eff.org/press/releases/eff-has-lavabits-back-con...
- dopylitty 2y ago
  In a way when people talk about encryption what they really mean is authorization to access data. The actual facts of whether or not data are encrypted using some whiz bang algorithm are irrelevant as long as some intermediary is ensuring that the data are only accessible by the intended clients. Sometimes I wonder if all the focus on encryption is actually wasting cycles that could be spent instead making sure the authorization model is bulletproof. For instance if a DBMS could ensure that only client A can access client A's data then does it matter if the data are stored encrypted in the DB? You might say, well if they aren't encrypted then anyone with root can just read the data directly but it may be the case that anyone with root will be able to access the data regardless of whether it's encrypted because they can just pull it from the memory space of the DB engine. There are a lot of considerations but it does seem like people get caught up in the "how" of encryption because of all the fancy maths and cool sounding algorithms rather than focusing on the "what" they're actually trying to accomplish which is usually "prevent clients from accessing data they shouldn't be able to access".
withinboredom 2y ago
I once reviewed a php library (don't remember which one but it was extremely popular) using `mb_strlen($string)` to get bytes to encrypt/decrypt. It was just waiting for someone to come along and en/decrypt with a non-english language.
- CiPHPerCoder 2y ago
  Was it similar to this? https://github.com/facebookarchive/php-graph-sdk/pull/552#issuecomment-182123879 https://github.com/facebookarchive/php-graph-sdk/pull/552#is...
  - withinboredom 2y ago
    That’s probably what they were going for, but no, it was just regular mb_stelen with a single arg.
cjonas 2y ago
Salesforce has an addon product called "Shield" that cost 30-50% of your overall license cost. It allows for encryption at rest at the field level (not all fields are supported and introduces query limitations). Companies purchase this add on thinking it makes them more secure, but it essentially does nothing to protect your data from exfiltration. If Salesforce's raw, multi-tenant data stores are leaked it seems unlikely that your company is going to be the ones taking heat for it. The only reason to go through this trouble is to check the regulatory box. Also it seems like Salesforce should be encrypting the entire disk at rest. Instead they created this feature to charge a premium to those with regulatory requires and try to shift the liability.
- jwnin 2y ago
  Markup aside, your description of what Salesforce is offering is what the article is saying should be done. Encryption of the disk at rest doesn't do anything for data exfil situations; it protects against physical theft or improper disposal - only.
  - cjonas 2y ago
    Sorry, still reading so having gotten there yet. How does Salesforce offering a "light" version of encryption at rest improve security? Or are you saying it's a better balance of performance / security by only selectively encrypting specific data points?
    - kbolino 2y ago
      It sounds like it is in addition to full-disk encryption, not instead of it. Encrypting each field with a distinct key that an attacker cannot glean by simply exfiltrating all the data on disk and/or all the data in RAM protects against online attacks in a way that full-disk encryption cannot. The real question is: does Salesforce do this properly?
      - apgwoz 2y ago
        It’s certainly possible that there’s a valid oversight here, but Salesforce has a rather talented security team, and the company truly lives by “Trust is our #1 value”^1 I can’t speak for the implementation, but my guess is that it’s been very thoroughly vetted by both internal security and external pen tests. They wouldn’t market a high profile security feature without that. (1: I am an ex-Heroku / Salesforce employee)
throwaway38375 2y ago
This might sound like a stupid question, but I'll ask anyway: What benefit does encryption at rest solve for something which is never intended to actually rest? For example, a MySQL database powering a web application is expected to be alive and responding to requests 24/7. It's never really intended to be at rest. So what benefit does encryption at rest bring? Won't a hacker be attempting to take data when it's online (and therefore not resting)?
- throwaway38375 2y ago
  I should clarify that I see the value of encryption at rest for something like an employee laptop, which could be left at a bar (while powered off) by accident. I just don't get the value of it for always online servers.
  - nine_k 2y ago
    An intruder gains access to an API box, and could try to read sensitive data from a DB. But the interesting fields are encrypted, and the key is somewhere in RAM. Not impossible to exfiltrate, but takes much longer time and more skill, thus cannot be made an unattended malware payload. Also, a key for one customer won't give access to data of other customers, even if the common database access credentials are obtained.
  - JackSlateur 2y ago
    How are online servers differents ? You can remove storage devices from online servers, without interruption. Said devices will contains data that could be "lost" that way. Hence: encryption at rest.
- nielsole 2y ago
  Your RAID reports an issue with one of the disks. You disable it and have the DC staff swap it. What happens to the disk? It might be resold. It might be put into the next server. It might land in an office drawer with a label "to be wiped".
  - 2y ago
SigmundA 2y ago
Full disk encryption or similarly transparent data encryption at the database allows you to continue to use the database as a full database. Decrypting on the "client" (app server) means you can't really use its native query language (SQL) effectively on the encrypted columns. Not sure what the state the art is in searchable encryption for db indexes, but just trying to do stuff that requires a scan becomes untenable due to having to read and decrypt on the client to find it or aggregate it. The security sandbox becomes app server memory instead of db server memory preventing effective use of the locality on the db server. I didn't see the article address this. Can make sense to encrypt specific sensitive columns that are not used for searches or aggregations later, but many systems the reason you have discrete data columns is to query them later not just retrieve a single record to decrypt and view on screen vs just storing a single document and encrypting / decrypting. Disk encryption is easy to use without reducing the functionality of the DB, client encryption specifically and purposely handicaps the functionality of the DB so its use case is very narrow IMO. I tend to treat both the app server and db ram as unencrypted so they require good access controls to use them (don't let Bob run sql queries against all the data unless he is authorized to do so).
- withinboredom 2y ago
  One way I’ve seen (eg, searching by zip code) is to encrypt all possible buckets you would search by (prefixes/suffixes) using a different (search) key, then encrypting the relationship foreign keys. Then the application searches for the encrypted values and decrypts the foreign keys.
  - kbolino 2y ago
    This strategy provides only obfuscation, not encryption. If the same plaintext always "encrypts" to the same ciphertext, it becomes possible (sometimes even trivial) for an attacker with access to large amounts of related information (such as the entire database) to use correlations and inference to effectively decipher it.
    - withinboredom 2y ago
Joe8Bit 2y ago
In my experience, a lot of the motivating factors for large enterprises mandating encryption at rest aren't about specific security controls. They will often hand wave in that direction, but as as the OP says in their post, without being able to describe a coherent threat model. Instead a lot of motivating factors I've seen are about preventing various paths for "legitimate" data disclosure to third parties. For example, when data at rest is combined with additional requirements like "bring your own key" it means a subpoena or NSL needs to be served on the _first party who owns the data_ (as they need to provide the keys) and can't be served on just the cloud provider without the first party having at least visibility of it.
chadsix 2y ago
> Then, if your server’s hard drive grows legs and walks out of the data center, your users’ most sensitive data will remain confidential. > Unfortunately, for the server-side encryption at rest use case, that’s basically all that Disk Encryption protects against. If you aren't able to self host, then encryption at rest is a real use case and the next best thing to actually controlling your data. That being said, obviously self hosting with FDE@Rest is the best. Or you can end up like the people who lost their data [1][2]. [1] https://spectrum.ieee.org/thousands-of-bitcoins-stolen-in-a-hack-on-linode https://spectrum.ieee.org/thousands-of-bitcoins-stolen-in-a-... [2] https://www.youtube.com/watch?v=g_JyDvBbZ6Q https://www.youtube.com/watch?v=g_JyDvBbZ6Q
- plingbang 2y ago
  > Or you can end up like the people who lost their data [1] I don't see how encryption at rest could've changed the outcome. In the article, the cloud provider, which has full control over the VMs, was compromised. The VMs were hosting various Bitcoin services, which needed continuous wallet access for operation. So, I'd say there was no data at rest to be secured. The attackers could theoretically patch the application to make malicious transactions or just extract the wallet from RAM. Also, the article suggests that the attackers were getting inside the running VMs rather than accessing VM storage directly.
JackSlateur 2y ago
This article is so narrow-minded .. I've work with physical servers in many compagnies. Every time, storage devices have a lifecycle : they are bought new, then plugged (and some data are written), then some times then move in another physical location, then they "die" (= put to the trash, either because they broke or for other reasons). Encryption at rest is an efficient way to secure data for the latter cases. The workers stole a hard disk ? No data is stolen. A hard disk is lost during transit, somehow ? No data is stolen. The device which is broken is retrieved by someone, opened-up, and being read-at directly ? No data is stolen. Your old device, put to the bin for some reasons, that could still be read if plugged on the proper hardware ? No data is stolen. All of that with little performance impact, and no software modification, few engineering overhead, very little work to do. It eases the lifecycle of storage devices, because storage devices are now worthless per-se (except for their physical cost, indeed). They carry virtually no data, no worth. Can you propose any other way to protect against those thread models ? Rewrite every software, so that every programs handle their own private keys ? Yeah, that's a nightmare, not gonna happen. And even if it did .. how would you encrypt your rootfs ? Ha yes : encryption at rest :)
- tossandthrow 2y ago
  For this thread model, the encryption should happen at the volume level, not application level. Which is also what the author write: >If you’re only interested in compliance requirements, you can probably just enable Full Disk Encryption and call it a day. Then, if your server’s hard drive grows legs and walks out of the data center, your users’ most sensitive data will remain confidential. > Unfortunately, for the server-side encryption at rest use case, that’s basically all that Disk Encryption protects against. (Something tells me that you did not read the article that is so narrow minded)
  - JackSlateur 2y ago
    I think I've written enough to illustrate that storage devices moves around without "growing legs and walk out" But yes, if the author drops all security thread related to encryption at rest, then encryption at rest is useless. I agree.
teeray 2y ago
Encryption at rest is nice for when a device has to get retired. Without the key, the drive is indistinguishable from random noise. No longer do you need to run DBAN for hours, put the drive through a degausser, or drill holes in it. No worrying about plaintext data hiding due to relocated sectors or wear-leveling. Just purge the keys and you’re done. Then the drive even has a chance at getting responsibly reused.
- aftbit 2y ago
  Yeah though SSDs make that even easier, with the aptly named SECURE ERASE command. Modern SSDs encrypt the contents of the drive at rest _anyway_ (transparently, using a key that's baked into the hardware) as encryption algorithms are very good at removing repeated patterns that might degrade the flash over time.
amluto 2y ago
An issue that wasn’t mentioned: protecting an encryption-at-rest key is not so easy, and the solutions I’ve heard of that are easy to deploy are barely effective against an attacker with physical access to the datacenter.
- hansvm 2y ago
  > Cryptography is a tool for turning a whole swathe of problems into key management problems. Key management problems are way harder than (virtually all) cryptographers think.
  - salawat 2y ago
    >Cryptography is a tool for turning a whole swathe of problems into key management problems. Key management problems are way harder than (virtually all) cryptographers think. As someone who has issues remembering where they left their house keys at times, I want this on a bloody coffee cup/t-shirt. Further, every practicing cryptographer everywhere should be forced to keep one on their person at all times, and be required to present it to do normal tasks as a reminder to them of the suffering their work creates for others implementing their cryptosystems. They probably won't care. But it'd make me feel better if they were as annoyed in the everyday as much as I am wrangling this kind of nonsense.
    - CiPHPerCoder 2y ago
      > As someone who has issues remembering where they left their house keys at times, I want this on a bloody coffee cup/t-shirt. I've heard this quote a lot over the years, but the first person who said it was probably Lea Kissner, the former CISO of Twitter. Here's their most recent post of the same statement: https://hachyderm.io/@leak/110784289970982813 https://hachyderm.io/@leak/110784289970982813
OptionOfT 2y ago
I'm a huge fan of MSSQL's transparent data encryption in situations where we have more than 1 client per database engine, and we want to separate the data physically. I worked on a project where separate databases (a la Postgres) tied to separate clients wasn't enough. Postgres still can read across the databases. With TDE we tie the key to the individual clients meaning even if the engine messes up, since the connection isn't made with the right key, you still can't read the contents. We still did encryption at rest as that comes for free these days, for reasons mentioned here in the comments. I just wish Postgres would come with TDE. Paying for software is fine, but the cost of MSSQL is way more than $0. In fact, it's usually cheaper to set up a Postgres instance per client. That way, when the engine messes up, well, there is only data of that client. And I know a database is unlikely to mess up. I'm more likely the culprit, and as such I prefer to have my guardrails.
- andyzweb 2y ago
  there have been a few attempts at adding TDE to postgresql. I think the cybertec patches are probably the most notable https://github.com/cybertec-postgresql/postgresql/tree/15tde/debian/patches https://github.com/cybertec-postgresql/postgresql/tree/15tde...
FooBarWidget 2y ago
Real life case study: We armed our servers (VPSes) to the teeth. Then an attacker gained access to our hosting provider's administration panel. They used that to download our hard disk content. Encryption at rest protects against this.
- tossandthrow 2y ago
  Encrypting your volumes would have mitigated this, and indeed you should have.
nightpool 2y ago
Encryption at rest is something your cloud provider does to pass SOC audits. End of sentence. If you have stronger security concerns, then you need to turn to other tools in your toolbox.
- ziddoap 2y ago
  Encryption at rest has several valid use-cases beyond SOC audits. End of sentence. Edit: Since this has gotten some negative votes, I'll happily expand. The two primary examples of FDE that are real-world useful (i.e. not just checking boxes) is loss of physical control of a device and cryptographic erasure (at device end-of-life). Neither of these use-cases in relevant to the threat model the article is discussing, but it's ridiculous to say that FDE is only for SOC.
  - nightpool 2y ago
    This is why I said "your cloud provider". If you're handling your own physical devices, yes, YMMV. (For example, FDE on company laptops should obviously be non-negotiable). But expecting it to do anything else is just magical thinking.
    - ziddoap 2y ago
      Cloud providers don't store stuff in a literal cloud, so it follows that they too must worry about their own physical devices. If you agree that FDE is good for physical access to lost/stolen devices and cryptographic erasure, I'm not sure why you don't think that applies to hardware in a data center which is just as capable as being lost/stolen, and also needs to be securely disposed of. >But expecting it to do anything else is just magical thinking. It certainly does more than just check boxes for SOC, which was my entire point.
deprave 2y ago
Encryption at Rest makes it easy to reason about data hygiene, since access to the data is gated through access to the keys. You want to delete data? Toss the keys. You want to confidentially process data? Make the keys available to a TEE or such. You want to prevent yourself from having constant access to the data? Let the client provide the keys. And of course, you want to protect the keys? Use an HSM.
motohagiography 2y ago
The main threat to encrypting health information is actually dishonest cryptographers. The quality of healthcare privacy reasoning is like, instead of searching for user Michael Knight in a database of cancer patients, you hash the name, use a bloom filter to determine if the hash exists in the protected data set or not, and then tell your regulators and the public that the foreign jurisdiction research assistants hired from fiverr don't have access to your health information because everything is encrypted and we only compare hashes. it's like that "sudo make me a sandwich," cartoon but, "cryptographically disclose your health information."
- CiPHPerCoder 2y ago
  > The main threat to encrypting health information is actually dishonest cryptographers. Wow, okay, you have me hooked. > instead of searching for user Michael Knight in a database of cancer patients, you hash the name, use a bloom filter to determine if the hash exists in the protected data set or not The protocol you loosely described here could be either totally fine or horrendously broken depending on the implementation details and architecture of the application. Not to mention the universal cryptography concerns; i.e., key management. > and then tell your regulators and the public that the foreign jurisdiction research assistants hired from fiverr don't have access to your health information because everything is encrypted and we only compare hashes. You've said "hashes" twice now, so I have to ask: 1. Are they only hashing, or are they using a keyed hash (e.g., HMAC)? 2. Are they doing something ridiculous like using MD5? If you're familiar with cryptography at all, passing around MD5(data) is vastly different from truncating HMAC-SHA384(data, staticKey) to a few bits and protecting staticKey with an HSM. Without more detail, I can't tell if your assertion that cryptographers are being dishonest is warranted. It's sort of like simplifying RSA encryption to "multiplying integers". Yes, that's what's happening on some level, but some essential information was omitted.
  - motohagiography 2y ago
    in your hmac example you have a separate key to manage, whereas in my example, it's implied it's sha256 or a variant that provides a layer of obfuscation in the lookup, and may even implement a "standard" to fool regulators. most regulations and standards say what tools to use (key bits, algos, etc) , and not that the implementations need to be approved by a professional. my example is that the scheme uses tools in a way that is meaningless because I can just take a phone book, hash the names, and check to see if they are in the cancer database as though it were an cleartext database. to say the data subject's data is private in this case because it is hashed (or often, inaccurately, "encrypted") is to mislead people who make decisions about it. I'm saying the designers of such systems represent themselves as cryptography and security experts who build weakened systems like this to mislead regulators on behalf of their bosses and users who just want the cleartext data. most protocols are a shell game where if you don't know where the root of trust secret is managed you're the sucker at the table. my experience has been that working cryptographers (protocol designers) in government, health, and financial institutions in general are a very refined class of bullshitters who pound the table whenever you ask them about details, and that one should not be intimidated by their theatrics. Hence in PHI management, dishonest cryptographers working on behalf of anti-privacy interests are the main threat.
talkingtab 2y ago
The author used the term "unknown unknowns". This is a variant of the way I talk about this state: "you don't know what you don't know". There are two audiences for this articles then, the ones who know what they don't know (or know it all), and those like me who are ignored-squared. I found it extremely helpful to help me "know what I don't know". If you have no idea what "encryption at rest" is or why it is important, then this is very useful and helpful. As the author clearly states it does not cover other things you don't know, like why and how to use full disk encryption. That said, it would be helpful for us i² [i squared] folks to have some of the more basic terms explained. Although "encryption at rest" is somewhat understandable, it would be pleasant to have it explained. For example what are the other kinds of encryption that are not "at rest"? There are a bunch of diligent amateurs out here, ones who know we don't know what we don't know and are attempting to build cryptographic things. We learn not to create our own implementations for example, but articles that specifically address us are good. [edit for clarity]
- CiPHPerCoder 2y ago
  > That said, it would be helpful for us i² [i squared] folks to have some of the more basic terms explained. Although "encryption at rest" is somewhat understandable, it would be pleasant to have it explained. That's helpful feedback, actually. What terms seemed opaque or misleading to you as you read it? I'm always happy to fix mistakes in blog posts to improve clarity. > For example what are the other kinds? I contrast "encryption at rest" with "encryption in transit" (i.e., TLS) and "end-to-end encryption" (E2EE for short; i.e., what Signal gives you).
  - throwaway81523 2y ago
    You're the author? Cool, I liked the article a lot. You do use a lot of terms that won't make sense to non-crypto nerds, like OAED, IND-CCA secure, etc. A lot of posters here are fixating too much on the "stolen hard disk" picture which I think the article addressed by declaring it out of scope. So the real points aren't getting through.
    - 2y ago
whartung 2y ago
Funny I always considered the impetus of Encryption at Rest to be the "left my laptop in the airport scenario". Or, maybe, "Where'd that CD with all of the medical records go?" Originally, I never felt that the EAR issue was that germane at data centers, since you're mostly protecting against someone backing through the loading door with a truck and stealing trays of drives. The disposal issue is valid, that was just something we were diligent with using a secure disposal service. Key management has always been an issue. Since no one wanted to be there when the machines spooled up after a glitch at 3am to type in a password to open the volumes. Everything else is basically locking the file cabinet drawers and taping the key to the back of it. Nowadays, it's different. Its more ubiquitous. Encrypting a laptop is a mouse click, and painless after that. Cloud providers have the infrastructure to manage it at their level. I'm still now sure what the solution is for a "self hosted" infrastructure. Hardware key modules are still pretty expensive. I haven't (not that I've looked at all recently) seen a writeup of how best to set of encryption on those 4 Dells my friend has racked across the country in Georgia (though even modern machines have some level of volume encryption I think).
- xenophonf 2y ago
  I keep meaning to deploy Mandos in my homelab: https://www.recompile.se/mandos https://www.recompile.se/mandos
bearjaws 2y ago
I wonder if OP works in healthcare, after HIPAA passed encryption at rest was the buzz word of the decade as it was one of the primary requirement of HIPAA. The problem of course being, most health care breaches are on applications that aren't at rest so all the data was being stolen anyway. From 2012-2021 I worked in health tech and on many calls with large customers and security questionnaires were on whether we were actually storing their data encrypted at rest. We even had to get audited for Aetna to validate encryption at rest (amongst other things). To me this seemed like such a joke of a requirement because all our data was in AWS, and breaches were far more likely from other avenues. So to me this reads as a jaded SWE or CISSP who has dealt with how much attention this one attack vector is paid, but ultimately it is kind of a given now in modern cloud infra.
- CiPHPerCoder 2y ago
  > I wonder if OP works in healthcare No, I work in applied cryptography. When I joined Amazon, the team I was hired on was called AWS Crypto Tools (which owned the AWS Encryption SDK, among other developer tools), while another team was called Transport Libraries (which owned S2N). When I left in 2023, they started adopting more of the "Encryption At Rest" lingo for what was previously called Crypto Tools. I don't know if they landed on different verbage since then. > after HIPAA passed encryption at rest was the buzz word of the decade as it was one of the primary requirement of HIPAA. Interesting. Thanks for sharing.
  - steelframe 2y ago
    It sounds like you left pretty close to when Greg did? What precipitated that, if you don't mind my asking?
    - CiPHPerCoder 2y ago
      Andy Jassy decided that it was time to Return To Office. I was hired in 2019 as a fully remote employee. This means my options in 2023 were a) move to Seattle or b) walk. I chose to walk. The leadership of the Cryptography org fought tooth and nail to get an exception for me, but were unable to do so. I still hold everyone in AWS Cryptography in high regard.
ozim 2y ago
I think reason is simple "encryption at rest" == "it is going to be encrypted in backup". People asking about "encryption at rest" are really asking if backups of your web application data are encrypted. Earlier I think it was quite a plague when just un-encrypted backup files were leaking out because someone exposed them on some open FTP to "quick and dirty" copy backups to new environment or to some test environment - and forgot to close it down or remove the files. Other threat would be developers exposing database server directly to the internet because someone from marketing wants to connect "new shiny super business intelligence" and developers not knowing better than "allow all" on firewall, then someone might steal raw db files but might not really have access to web application and encryption keys. For the reasons mentioned by author I can see how it seems like security theater. But I think my reasons are quite valid and on topic.
- CiPHPerCoder 2y ago
  The thing that's security theater isn't encrypting at rest in general. The thing that's security theater is encrypting insecurely, or failing to authenticate your access patterns, such that an attacker with privileged access to your database hardware can realistically get all the plaintext records they want.
  - indymike 2y ago
    >> The thing that's security theater isn't encrypting at rest in general > The thing that's security theater is encrypting insecurely Security theater should be defined as: Doing things that outwardly appear to improve security but have de minimus or less effect on actual security. The 93 section questionnaire from bigco's IT department is security theater. Filling it out does zero to improve security for bigco or myco or my users.
    - CiPHPerCoder 2y ago
      > Doing things that outwardly appear to improve security but have de minimus or less effect on actual security. Right. And that's exactly the situation the article describes. The accusation of "security theater" was only levied when IT departments reached for the "full disk encryption" potion to mitigate the ailment of "attacker has active, online access to our database via SQL injection", when that's not at all what it's designed to prevent. They can insist that they're "encrypting their database", but does it actually matter for the threats they're worried about? No. Thus, security theater. The same is true of insecure client-side encryption.
fulafel 2y ago
> The first question to answer when data is being encrypted is, “How are the keys being managed?” This is a very deep rabbit hole of complexity, but one good answer for a centralized service is, “Cloud-based key management service with audit logging”; i.e. AWS KMS, Google CloudKMS, etc. This is of course the beef. What's the best practice in managing user data keys so that data is available only when there's an authenticated user around? There are ways to derive keys from the secret exchange involved in user authentication.
- CiPHPerCoder 2y ago
  > What's the best practice in managing user data keys so that data is available only when there's an authenticated user around? What does it mean for an authenticated user to be "around"? If you want a human to manually approve the decryption operations of the machine, but it can still store/encrypt new records, you can use HPKE so that only the person possessing the corresponding secret key can decipher the data. At least, you can until a quantum computer is built.
  - justincormack 2y ago
    “Around” usually means proving possession of a signing key on a connection.
  - fulafel 2y ago
    A working definition for some apps could be: The user's data should not be available to the system if there isn't an active user session, such that the user's privacy interests are cryptographically protected in event of a breach or data leak occurring when the user is not actively using the system. I wasn't thinking of manual approval of any cryptographic steps. Just that when you log in to work on your data stored in the system, the system can only then decrypt the data, and when you log out, the system forgets the keys until next time. It all depends on the type of app of course.
    - CiPHPerCoder 2y ago
      Okay, this sounds vaguely like a problem that may be solved by "HPKE where the secret key is reconstructed from a threshold secret sharing scheme" (>=2 of N shares needed, 1 held by the service and 1 held by the employee's hardware device, where 1 additional share is held in cold storage for break-glass reasons). I would need to actually sit down and walk through the architecture, threat model, etc. to recommend anything specific. I'm not going to do that on a message board comment, because I probably am missing something.
jnwatson 2y ago
The article has some great points but has an unfortunate title that has lead to an argument the author isn't making. The cryptographic details about preventing confused deputy are excellent and worth studying.
rsync 2y ago
I've read the article and the entire comment thread and nobody is talking about the cloud provider itself ... ? When someone borgs[1] up their data to store at rsync.net we just assume that we are the threat. Of course I don't believe that but it's perfectly rational and we encourage people to think of rsync.net as a threat as they design their backups. Comments in this thread are actually discounting the threat of Amazon personnel, GCS personnel, etc., as if that threat was zero. Not only is it non-zero, I would go further: if you're storing the data on AWS and generating your keys on AWS and managing your keys with AWS ... you're doing it wrong. [1] https://www.borgbackup.org/ https://www.borgbackup.org/
- CiPHPerCoder 2y ago
  This is an interesting and important point that you raised. > if you're storing the data on AWS and generating your keys on AWS and managing your keys with AWS ... you're doing it wrong. This is a reasonable thing to do if you've decided that you trust AWS and expect any violations of this trust to be dealt with by the legal department. It's less reasonable if you're concerned about AWS employees going rogue and somehow breaking the security of KMS without anyone else knowing. It's even less reasonable to do this if you're concerned about AWS credentials being leaked or compromised, which in turn grants an attacker access to KMS (i.e., a government would be more successful by compelling IAM to grant access than they would trying to subpoena KMS for raw keys). (Sure, you can audit access via CloudTrail, but that's a forensics tool, not a prevention tool.) But that's kind of the point I wrote in the article, no? You need to know your threat model. You've stated yours succinctly, and I think it's a commendable one, but many enterprises are a bit more relaxed.
  - jjav 2y ago
    > It's less reasonable if you're concerned about AWS employees going rogue and somehow breaking the security of KMS without anyone else knowing. That's the least of the concerns. Remember AWS is subject to court orders of all types (legitimate ones and NSLs). Even if nobody goes rogue, any data that AWS (or any cloud/SaaS provide) could access, must be assumed to be compromised.
cedws 2y ago
In the context of cloud, it's cargo cult security. Something somewhere says "you must have encryption at rest." I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. So what are you protecting again? If you're storing the decryption keys in KMS in the same cloud, you're not hiding anything from the cloud provider. The only rationale I can think of for doing this is defense-in-depth, but seeing how many companies struggle to even get IAM right I doubt this would help much. Security compliance and real world security are a universe apart. You have encryption at rest with mandated AES-GCM/SHA512? Cool story bro, some teenagers just broke into your network with a bit of social engineering and a 6 year old CVE.
- snowstormsun 2y ago
  > I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. I find that very easy to belive.
- CiPHPerCoder 2y ago
  > I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. I am confident that they do. Even better, they can be configured to use your KMS key rather than the service key, and you can configure KMS to use external key stores (i.e., an HSM in your datacenter outside of AWS's control, that you could theoretically pull the plug on at any time).
wg0 2y ago
On the subject of Encryption ar Rest: On AWS RDS, you can't turn off encryption once you turn it on. The encryption secret keys always stay with AWS and all you can download are the public keys. So I'm not so sure what's the point of encryption at rest in AWS except just to tick off a compliance and regulatory checklist. The private key is with them anyway, just don't encrypt and save few milliwatts of power.
- CiPHPerCoder 2y ago
  > So I'm not so sure what's the point of encryption at rest in AWS except just to tick off a compliance and regulatory checklist. > The private key is with them anyway, just don't encrypt and save few milliwatts of power. "Them" is Amazon, a company with over 1 million employees, last I checked. It's perfectly reasonable to trust the KMS team to keep your keys secure, even if you don't trust the RDS team to never try to look at your data. I know it's tempting to think of all of AWS as a sort of "Dave" who wears multiple hats, but we're talking about a large company. Protecting against other parts of the same company is still a worthwhile and meaningful security control.
  - wg0 2y ago
    As a customer, I don't know neither I do care how they have teamed up internally. Not my problem. From my perspective, the secret keys I don't have. Just AWS has and they can decrypt whatever and whenever they want maybe because they have a warrant or some three letter agency has them do it.
  - outworlder 2y ago
    > It's perfectly reasonable to trust the KMS team to keep your keys secure, even if you don't trust the RDS team to never try to look at your data. If the database is live, then the data is able to be decrypted and who knows where it ends up. Encryption at rest solves only the threat scenario where the RDS team has access to the database storage layer. It doesn't do anything to mitigate any threats after it has been read from storage.

199 points by chillax 2y ago

Encryption at rest is misleading. It is often used to describe an insecure situation consisting of disk encryption only instead of encrypted data store values.

thom 2y ago
The article explains why the former is often just security theatre.
- tasn 2y ago
  Calling it security theater is a bit harsh, though it's definitely not a cure all solution. It does protect against physically taking of and improper disposal of HDs when done correctly. Which can be quite a significant vector.
  - klabb3 2y ago
    It’s not security theatre in the rare case that the stated goals is to protect against physical attacks, or for hardware disposal. If the goal is to put “military grade encryption” sticker in the sales deck, or to pass some certification, or to create a vague cloud of plausibility that you’re taking security seriously in some other way, then absolutely it’s security theatre. It deserves to be called out.
    - jeffbee 2y ago
      It's not "theater" if the customer requires that certification.
      - mbb70 2y ago
        If the customer wants a show, give them a show. Doesn't make it not theater.
doubled112 2y ago
Encryption at rest is great if I forget my laptop on a bus. Reduces it to a VISA problem. But who is stealing data off of servers by taking the server? Maybe it saves you when you dispose of the disks? Maybe a home server in a break in? This has always seemed obvious to me.
- zsims 2y ago
  There are other paths to the attack he mentioned. Eg you find an API that accepts ciphertext or part of. Or a cloud backup/restore flow. Likely you need another vulnerability but it does happen.
- erinnh 2y ago
  I can think of two reasons: - Physically moving servers outside of the cages they are supposed to be in. (inside/outside DC) - Easier to dispose of disks (repair/EOL)
- dspillett 2y ago
  > But who is stealing data off of servers by taking the server? Away from pure bare metal, in the presence of bugs in security separation someone could steal data simply by having a VM on the same server as yours. Encryption at-rest with each VM or service/account having their own keys removes the risk of data accidentally becoming visible in another environment that is sharing the hardware. Though the chance of some useful data being revealed this way is pretty small on large scale cloud installations, and if you are using a cheap VPS host for sensitive data you might not be valuing your data enough.
- immibis 2y ago
  It has happened before that disks have been taken from servers in third-party data centers. Most commonly by the feds, but also by other actors. Do you control the physical security of your server?

candiddevmike 2y ago

This blog post is a lot of words to say encryption != authentication.

photochemsyn 2y ago
I think it's very valuable for people who might not already now this to see an example of why it's true, as the post does: > "What’s happening here is simple: The web application has the ability to decrypt different records encrypted with different keys. If you pass records that were encrypted for Alice to the application to decrypt it for Bob, and you’re not authenticating your access patterns, Bob can read Alice’s data by performing this attack." An interesting question here is whether or not there's a god key that allows the administrator to decrypt all the data even if they can't authenticate as the user (or if they just have copies of all keys). Searching HN for 'lavabits' turns up some results related to this, e.g. https://www.eff.org/press/releases/eff-has-lavabits-back-contempt-court-appeal https://www.eff.org/press/releases/eff-has-lavabits-back-con...
dopylitty 2y ago
In a way when people talk about encryption what they really mean is authorization to access data. The actual facts of whether or not data are encrypted using some whiz bang algorithm are irrelevant as long as some intermediary is ensuring that the data are only accessible by the intended clients. Sometimes I wonder if all the focus on encryption is actually wasting cycles that could be spent instead making sure the authorization model is bulletproof. For instance if a DBMS could ensure that only client A can access client A's data then does it matter if the data are stored encrypted in the DB? You might say, well if they aren't encrypted then anyone with root can just read the data directly but it may be the case that anyone with root will be able to access the data regardless of whether it's encrypted because they can just pull it from the memory space of the DB engine. There are a lot of considerations but it does seem like people get caught up in the "how" of encryption because of all the fancy maths and cool sounding algorithms rather than focusing on the "what" they're actually trying to accomplish which is usually "prevent clients from accessing data they shouldn't be able to access".

withinboredom 2y ago

I once reviewed a php library (don't remember which one but it was extremely popular) using `mb_strlen($string)` to get bytes to encrypt/decrypt. It was just waiting for someone to come along and en/decrypt with a non-english language.

CiPHPerCoder 2y ago
Was it similar to this? https://github.com/facebookarchive/php-graph-sdk/pull/552#issuecomment-182123879 https://github.com/facebookarchive/php-graph-sdk/pull/552#is...
- withinboredom 2y ago
  That’s probably what they were going for, but no, it was just regular mb_stelen with a single arg.

cjonas 2y ago

Salesforce has an addon product called "Shield" that cost 30-50% of your overall license cost. It allows for encryption at rest at the field level (not all fields are supported and introduces query limitations). Companies purchase this add on thinking it makes them more secure, but it essentially does nothing to protect your data from exfiltration. If Salesforce's raw, multi-tenant data stores are leaked it seems unlikely that your company is going to be the ones taking heat for it. The only reason to go through this trouble is to check the regulatory box. Also it seems like Salesforce should be encrypting the entire disk at rest. Instead they created this feature to charge a premium to those with regulatory requires and try to shift the liability.

jwnin 2y ago
Markup aside, your description of what Salesforce is offering is what the article is saying should be done. Encryption of the disk at rest doesn't do anything for data exfil situations; it protects against physical theft or improper disposal - only.
- cjonas 2y ago
  Sorry, still reading so having gotten there yet. How does Salesforce offering a "light" version of encryption at rest improve security? Or are you saying it's a better balance of performance / security by only selectively encrypting specific data points?
  - kbolino 2y ago
    It sounds like it is in addition to full-disk encryption, not instead of it. Encrypting each field with a distinct key that an attacker cannot glean by simply exfiltrating all the data on disk and/or all the data in RAM protects against online attacks in a way that full-disk encryption cannot. The real question is: does Salesforce do this properly?
    - apgwoz 2y ago
      It’s certainly possible that there’s a valid oversight here, but Salesforce has a rather talented security team, and the company truly lives by “Trust is our #1 value”^1 I can’t speak for the implementation, but my guess is that it’s been very thoroughly vetted by both internal security and external pen tests. They wouldn’t market a high profile security feature without that. (1: I am an ex-Heroku / Salesforce employee)

throwaway38375 2y ago

This might sound like a stupid question, but I'll ask anyway: What benefit does encryption at rest solve for something which is never intended to actually rest? For example, a MySQL database powering a web application is expected to be alive and responding to requests 24/7. It's never really intended to be at rest. So what benefit does encryption at rest bring? Won't a hacker be attempting to take data when it's online (and therefore not resting)?

throwaway38375 2y ago
I should clarify that I see the value of encryption at rest for something like an employee laptop, which could be left at a bar (while powered off) by accident. I just don't get the value of it for always online servers.
- nine_k 2y ago
  An intruder gains access to an API box, and could try to read sensitive data from a DB. But the interesting fields are encrypted, and the key is somewhere in RAM. Not impossible to exfiltrate, but takes much longer time and more skill, thus cannot be made an unattended malware payload. Also, a key for one customer won't give access to data of other customers, even if the common database access credentials are obtained.
- JackSlateur 2y ago
  How are online servers differents ? You can remove storage devices from online servers, without interruption. Said devices will contains data that could be "lost" that way. Hence: encryption at rest.
nielsole 2y ago
Your RAID reports an issue with one of the disks. You disable it and have the DC staff swap it. What happens to the disk? It might be resold. It might be put into the next server. It might land in an office drawer with a label "to be wiped".
- 2y ago

SigmundA 2y ago

Full disk encryption or similarly transparent data encryption at the database allows you to continue to use the database as a full database. Decrypting on the "client" (app server) means you can't really use its native query language (SQL) effectively on the encrypted columns. Not sure what the state the art is in searchable encryption for db indexes, but just trying to do stuff that requires a scan becomes untenable due to having to read and decrypt on the client to find it or aggregate it. The security sandbox becomes app server memory instead of db server memory preventing effective use of the locality on the db server. I didn't see the article address this. Can make sense to encrypt specific sensitive columns that are not used for searches or aggregations later, but many systems the reason you have discrete data columns is to query them later not just retrieve a single record to decrypt and view on screen vs just storing a single document and encrypting / decrypting. Disk encryption is easy to use without reducing the functionality of the DB, client encryption specifically and purposely handicaps the functionality of the DB so its use case is very narrow IMO. I tend to treat both the app server and db ram as unencrypted so they require good access controls to use them (don't let Bob run sql queries against all the data unless he is authorized to do so).

withinboredom 2y ago
One way I’ve seen (eg, searching by zip code) is to encrypt all possible buckets you would search by (prefixes/suffixes) using a different (search) key, then encrypting the relationship foreign keys. Then the application searches for the encrypted values and decrypts the foreign keys.
- kbolino 2y ago
  This strategy provides only obfuscation, not encryption. If the same plaintext always "encrypts" to the same ciphertext, it becomes possible (sometimes even trivial) for an attacker with access to large amounts of related information (such as the entire database) to use correlations and inference to effectively decipher it.
  - withinboredom 2y ago

Joe8Bit 2y ago

In my experience, a lot of the motivating factors for large enterprises mandating encryption at rest aren't about specific security controls. They will often hand wave in that direction, but as as the OP says in their post, without being able to describe a coherent threat model. Instead a lot of motivating factors I've seen are about preventing various paths for "legitimate" data disclosure to third parties. For example, when data at rest is combined with additional requirements like "bring your own key" it means a subpoena or NSL needs to be served on the _first party who owns the data_ (as they need to provide the keys) and can't be served on just the cloud provider without the first party having at least visibility of it.

chadsix 2y ago

> Then, if your server’s hard drive grows legs and walks out of the data center, your users’ most sensitive data will remain confidential. > Unfortunately, for the server-side encryption at rest use case, that’s basically all that Disk Encryption protects against. If you aren't able to self host, then encryption at rest is a real use case and the next best thing to actually controlling your data. That being said, obviously self hosting with FDE@Rest is the best. Or you can end up like the people who lost their data [1][2]. [1] https://spectrum.ieee.org/thousands-of-bitcoins-stolen-in-a-hack-on-linode https://spectrum.ieee.org/thousands-of-bitcoins-stolen-in-a-... [2] https://www.youtube.com/watch?v=g_JyDvBbZ6Q https://www.youtube.com/watch?v=g_JyDvBbZ6Q

plingbang 2y ago
> Or you can end up like the people who lost their data [1] I don't see how encryption at rest could've changed the outcome. In the article, the cloud provider, which has full control over the VMs, was compromised. The VMs were hosting various Bitcoin services, which needed continuous wallet access for operation. So, I'd say there was no data at rest to be secured. The attackers could theoretically patch the application to make malicious transactions or just extract the wallet from RAM. Also, the article suggests that the attackers were getting inside the running VMs rather than accessing VM storage directly.

JackSlateur 2y ago

This article is so narrow-minded .. I've work with physical servers in many compagnies. Every time, storage devices have a lifecycle : they are bought new, then plugged (and some data are written), then some times then move in another physical location, then they "die" (= put to the trash, either because they broke or for other reasons). Encryption at rest is an efficient way to secure data for the latter cases. The workers stole a hard disk ? No data is stolen. A hard disk is lost during transit, somehow ? No data is stolen. The device which is broken is retrieved by someone, opened-up, and being read-at directly ? No data is stolen. Your old device, put to the bin for some reasons, that could still be read if plugged on the proper hardware ? No data is stolen. All of that with little performance impact, and no software modification, few engineering overhead, very little work to do. It eases the lifecycle of storage devices, because storage devices are now worthless per-se (except for their physical cost, indeed). They carry virtually no data, no worth. Can you propose any other way to protect against those thread models ? Rewrite every software, so that every programs handle their own private keys ? Yeah, that's a nightmare, not gonna happen. And even if it did .. how would you encrypt your rootfs ? Ha yes : encryption at rest :)

tossandthrow 2y ago
For this thread model, the encryption should happen at the volume level, not application level. Which is also what the author write: >If you’re only interested in compliance requirements, you can probably just enable Full Disk Encryption and call it a day. Then, if your server’s hard drive grows legs and walks out of the data center, your users’ most sensitive data will remain confidential. > Unfortunately, for the server-side encryption at rest use case, that’s basically all that Disk Encryption protects against. (Something tells me that you did not read the article that is so narrow minded)
- JackSlateur 2y ago
  I think I've written enough to illustrate that storage devices moves around without "growing legs and walk out" But yes, if the author drops all security thread related to encryption at rest, then encryption at rest is useless. I agree.

teeray 2y ago

Encryption at rest is nice for when a device has to get retired. Without the key, the drive is indistinguishable from random noise. No longer do you need to run DBAN for hours, put the drive through a degausser, or drill holes in it. No worrying about plaintext data hiding due to relocated sectors or wear-leveling. Just purge the keys and you’re done. Then the drive even has a chance at getting responsibly reused.

aftbit 2y ago
Yeah though SSDs make that even easier, with the aptly named SECURE ERASE command. Modern SSDs encrypt the contents of the drive at rest _anyway_ (transparently, using a key that's baked into the hardware) as encryption algorithms are very good at removing repeated patterns that might degrade the flash over time.

amluto 2y ago

An issue that wasn’t mentioned: protecting an encryption-at-rest key is not so easy, and the solutions I’ve heard of that are easy to deploy are barely effective against an attacker with physical access to the datacenter.

hansvm 2y ago
> Cryptography is a tool for turning a whole swathe of problems into key management problems. Key management problems are way harder than (virtually all) cryptographers think.
- salawat 2y ago
  >Cryptography is a tool for turning a whole swathe of problems into key management problems. Key management problems are way harder than (virtually all) cryptographers think. As someone who has issues remembering where they left their house keys at times, I want this on a bloody coffee cup/t-shirt. Further, every practicing cryptographer everywhere should be forced to keep one on their person at all times, and be required to present it to do normal tasks as a reminder to them of the suffering their work creates for others implementing their cryptosystems. They probably won't care. But it'd make me feel better if they were as annoyed in the everyday as much as I am wrangling this kind of nonsense.
  - CiPHPerCoder 2y ago
    > As someone who has issues remembering where they left their house keys at times, I want this on a bloody coffee cup/t-shirt. I've heard this quote a lot over the years, but the first person who said it was probably Lea Kissner, the former CISO of Twitter. Here's their most recent post of the same statement: https://hachyderm.io/@leak/110784289970982813 https://hachyderm.io/@leak/110784289970982813

OptionOfT 2y ago

I'm a huge fan of MSSQL's transparent data encryption in situations where we have more than 1 client per database engine, and we want to separate the data physically. I worked on a project where separate databases (a la Postgres) tied to separate clients wasn't enough. Postgres still can read across the databases. With TDE we tie the key to the individual clients meaning even if the engine messes up, since the connection isn't made with the right key, you still can't read the contents. We still did encryption at rest as that comes for free these days, for reasons mentioned here in the comments. I just wish Postgres would come with TDE. Paying for software is fine, but the cost of MSSQL is way more than $0. In fact, it's usually cheaper to set up a Postgres instance per client. That way, when the engine messes up, well, there is only data of that client. And I know a database is unlikely to mess up. I'm more likely the culprit, and as such I prefer to have my guardrails.

andyzweb 2y ago
there have been a few attempts at adding TDE to postgresql. I think the cybertec patches are probably the most notable https://github.com/cybertec-postgresql/postgresql/tree/15tde/debian/patches https://github.com/cybertec-postgresql/postgresql/tree/15tde...

FooBarWidget 2y ago

Real life case study: We armed our servers (VPSes) to the teeth. Then an attacker gained access to our hosting provider's administration panel. They used that to download our hard disk content. Encryption at rest protects against this.

tossandthrow 2y ago
Encrypting your volumes would have mitigated this, and indeed you should have.

nightpool 2y ago

Encryption at rest is something your cloud provider does to pass SOC audits. End of sentence. If you have stronger security concerns, then you need to turn to other tools in your toolbox.

ziddoap 2y ago
Encryption at rest has several valid use-cases beyond SOC audits. End of sentence. Edit: Since this has gotten some negative votes, I'll happily expand. The two primary examples of FDE that are real-world useful (i.e. not just checking boxes) is loss of physical control of a device and cryptographic erasure (at device end-of-life). Neither of these use-cases in relevant to the threat model the article is discussing, but it's ridiculous to say that FDE is only for SOC.
- nightpool 2y ago
  This is why I said "your cloud provider". If you're handling your own physical devices, yes, YMMV. (For example, FDE on company laptops should obviously be non-negotiable). But expecting it to do anything else is just magical thinking.
  - ziddoap 2y ago
    Cloud providers don't store stuff in a literal cloud, so it follows that they too must worry about their own physical devices. If you agree that FDE is good for physical access to lost/stolen devices and cryptographic erasure, I'm not sure why you don't think that applies to hardware in a data center which is just as capable as being lost/stolen, and also needs to be securely disposed of. >But expecting it to do anything else is just magical thinking. It certainly does more than just check boxes for SOC, which was my entire point.

deprave 2y ago

Encryption at Rest makes it easy to reason about data hygiene, since access to the data is gated through access to the keys. You want to delete data? Toss the keys. You want to confidentially process data? Make the keys available to a TEE or such. You want to prevent yourself from having constant access to the data? Let the client provide the keys. And of course, you want to protect the keys? Use an HSM.

motohagiography 2y ago

The main threat to encrypting health information is actually dishonest cryptographers. The quality of healthcare privacy reasoning is like, instead of searching for user Michael Knight in a database of cancer patients, you hash the name, use a bloom filter to determine if the hash exists in the protected data set or not, and then tell your regulators and the public that the foreign jurisdiction research assistants hired from fiverr don't have access to your health information because everything is encrypted and we only compare hashes. it's like that "sudo make me a sandwich," cartoon but, "cryptographically disclose your health information."

CiPHPerCoder 2y ago
> The main threat to encrypting health information is actually dishonest cryptographers. Wow, okay, you have me hooked. > instead of searching for user Michael Knight in a database of cancer patients, you hash the name, use a bloom filter to determine if the hash exists in the protected data set or not The protocol you loosely described here could be either totally fine or horrendously broken depending on the implementation details and architecture of the application. Not to mention the universal cryptography concerns; i.e., key management. > and then tell your regulators and the public that the foreign jurisdiction research assistants hired from fiverr don't have access to your health information because everything is encrypted and we only compare hashes. You've said "hashes" twice now, so I have to ask: 1. Are they only hashing, or are they using a keyed hash (e.g., HMAC)? 2. Are they doing something ridiculous like using MD5? If you're familiar with cryptography at all, passing around MD5(data) is vastly different from truncating HMAC-SHA384(data, staticKey) to a few bits and protecting staticKey with an HSM. Without more detail, I can't tell if your assertion that cryptographers are being dishonest is warranted. It's sort of like simplifying RSA encryption to "multiplying integers". Yes, that's what's happening on some level, but some essential information was omitted.
- motohagiography 2y ago
  in your hmac example you have a separate key to manage, whereas in my example, it's implied it's sha256 or a variant that provides a layer of obfuscation in the lookup, and may even implement a "standard" to fool regulators. most regulations and standards say what tools to use (key bits, algos, etc) , and not that the implementations need to be approved by a professional. my example is that the scheme uses tools in a way that is meaningless because I can just take a phone book, hash the names, and check to see if they are in the cancer database as though it were an cleartext database. to say the data subject's data is private in this case because it is hashed (or often, inaccurately, "encrypted") is to mislead people who make decisions about it. I'm saying the designers of such systems represent themselves as cryptography and security experts who build weakened systems like this to mislead regulators on behalf of their bosses and users who just want the cleartext data. most protocols are a shell game where if you don't know where the root of trust secret is managed you're the sucker at the table. my experience has been that working cryptographers (protocol designers) in government, health, and financial institutions in general are a very refined class of bullshitters who pound the table whenever you ask them about details, and that one should not be intimidated by their theatrics. Hence in PHI management, dishonest cryptographers working on behalf of anti-privacy interests are the main threat.

talkingtab 2y ago

The author used the term "unknown unknowns". This is a variant of the way I talk about this state: "you don't know what you don't know". There are two audiences for this articles then, the ones who know what they don't know (or know it all), and those like me who are ignored-squared. I found it extremely helpful to help me "know what I don't know". If you have no idea what "encryption at rest" is or why it is important, then this is very useful and helpful. As the author clearly states it does not cover other things you don't know, like why and how to use full disk encryption. That said, it would be helpful for us i² [i squared] folks to have some of the more basic terms explained. Although "encryption at rest" is somewhat understandable, it would be pleasant to have it explained. For example what are the other kinds of encryption that are not "at rest"? There are a bunch of diligent amateurs out here, ones who know we don't know what we don't know and are attempting to build cryptographic things. We learn not to create our own implementations for example, but articles that specifically address us are good. [edit for clarity]

CiPHPerCoder 2y ago
> That said, it would be helpful for us i² [i squared] folks to have some of the more basic terms explained. Although "encryption at rest" is somewhat understandable, it would be pleasant to have it explained. That's helpful feedback, actually. What terms seemed opaque or misleading to you as you read it? I'm always happy to fix mistakes in blog posts to improve clarity. > For example what are the other kinds? I contrast "encryption at rest" with "encryption in transit" (i.e., TLS) and "end-to-end encryption" (E2EE for short; i.e., what Signal gives you).
- throwaway81523 2y ago
  You're the author? Cool, I liked the article a lot. You do use a lot of terms that won't make sense to non-crypto nerds, like OAED, IND-CCA secure, etc. A lot of posters here are fixating too much on the "stolen hard disk" picture which I think the article addressed by declaring it out of scope. So the real points aren't getting through.
  - 2y ago

whartung 2y ago

Funny I always considered the impetus of Encryption at Rest to be the "left my laptop in the airport scenario". Or, maybe, "Where'd that CD with all of the medical records go?" Originally, I never felt that the EAR issue was that germane at data centers, since you're mostly protecting against someone backing through the loading door with a truck and stealing trays of drives. The disposal issue is valid, that was just something we were diligent with using a secure disposal service. Key management has always been an issue. Since no one wanted to be there when the machines spooled up after a glitch at 3am to type in a password to open the volumes. Everything else is basically locking the file cabinet drawers and taping the key to the back of it. Nowadays, it's different. Its more ubiquitous. Encrypting a laptop is a mouse click, and painless after that. Cloud providers have the infrastructure to manage it at their level. I'm still now sure what the solution is for a "self hosted" infrastructure. Hardware key modules are still pretty expensive. I haven't (not that I've looked at all recently) seen a writeup of how best to set of encryption on those 4 Dells my friend has racked across the country in Georgia (though even modern machines have some level of volume encryption I think).

xenophonf 2y ago
I keep meaning to deploy Mandos in my homelab: https://www.recompile.se/mandos https://www.recompile.se/mandos

bearjaws 2y ago

I wonder if OP works in healthcare, after HIPAA passed encryption at rest was the buzz word of the decade as it was one of the primary requirement of HIPAA. The problem of course being, most health care breaches are on applications that aren't at rest so all the data was being stolen anyway. From 2012-2021 I worked in health tech and on many calls with large customers and security questionnaires were on whether we were actually storing their data encrypted at rest. We even had to get audited for Aetna to validate encryption at rest (amongst other things). To me this seemed like such a joke of a requirement because all our data was in AWS, and breaches were far more likely from other avenues. So to me this reads as a jaded SWE or CISSP who has dealt with how much attention this one attack vector is paid, but ultimately it is kind of a given now in modern cloud infra.

CiPHPerCoder 2y ago
> I wonder if OP works in healthcare No, I work in applied cryptography. When I joined Amazon, the team I was hired on was called AWS Crypto Tools (which owned the AWS Encryption SDK, among other developer tools), while another team was called Transport Libraries (which owned S2N). When I left in 2023, they started adopting more of the "Encryption At Rest" lingo for what was previously called Crypto Tools. I don't know if they landed on different verbage since then. > after HIPAA passed encryption at rest was the buzz word of the decade as it was one of the primary requirement of HIPAA. Interesting. Thanks for sharing.
- steelframe 2y ago
  It sounds like you left pretty close to when Greg did? What precipitated that, if you don't mind my asking?
  - CiPHPerCoder 2y ago
    Andy Jassy decided that it was time to Return To Office. I was hired in 2019 as a fully remote employee. This means my options in 2023 were a) move to Seattle or b) walk. I chose to walk. The leadership of the Cryptography org fought tooth and nail to get an exception for me, but were unable to do so. I still hold everyone in AWS Cryptography in high regard.

ozim 2y ago

I think reason is simple "encryption at rest" == "it is going to be encrypted in backup". People asking about "encryption at rest" are really asking if backups of your web application data are encrypted. Earlier I think it was quite a plague when just un-encrypted backup files were leaking out because someone exposed them on some open FTP to "quick and dirty" copy backups to new environment or to some test environment - and forgot to close it down or remove the files. Other threat would be developers exposing database server directly to the internet because someone from marketing wants to connect "new shiny super business intelligence" and developers not knowing better than "allow all" on firewall, then someone might steal raw db files but might not really have access to web application and encryption keys. For the reasons mentioned by author I can see how it seems like security theater. But I think my reasons are quite valid and on topic.

CiPHPerCoder 2y ago
The thing that's security theater isn't encrypting at rest in general. The thing that's security theater is encrypting insecurely, or failing to authenticate your access patterns, such that an attacker with privileged access to your database hardware can realistically get all the plaintext records they want.
- indymike 2y ago
  >> The thing that's security theater isn't encrypting at rest in general > The thing that's security theater is encrypting insecurely Security theater should be defined as: Doing things that outwardly appear to improve security but have de minimus or less effect on actual security. The 93 section questionnaire from bigco's IT department is security theater. Filling it out does zero to improve security for bigco or myco or my users.
  - CiPHPerCoder 2y ago
    > Doing things that outwardly appear to improve security but have de minimus or less effect on actual security. Right. And that's exactly the situation the article describes. The accusation of "security theater" was only levied when IT departments reached for the "full disk encryption" potion to mitigate the ailment of "attacker has active, online access to our database via SQL injection", when that's not at all what it's designed to prevent. They can insist that they're "encrypting their database", but does it actually matter for the threats they're worried about? No. Thus, security theater. The same is true of insecure client-side encryption.

fulafel 2y ago

> The first question to answer when data is being encrypted is, “How are the keys being managed?” This is a very deep rabbit hole of complexity, but one good answer for a centralized service is, “Cloud-based key management service with audit logging”; i.e. AWS KMS, Google CloudKMS, etc. This is of course the beef. What's the best practice in managing user data keys so that data is available only when there's an authenticated user around? There are ways to derive keys from the secret exchange involved in user authentication.

CiPHPerCoder 2y ago
> What's the best practice in managing user data keys so that data is available only when there's an authenticated user around? What does it mean for an authenticated user to be "around"? If you want a human to manually approve the decryption operations of the machine, but it can still store/encrypt new records, you can use HPKE so that only the person possessing the corresponding secret key can decipher the data. At least, you can until a quantum computer is built.
- justincormack 2y ago
  “Around” usually means proving possession of a signing key on a connection.
- fulafel 2y ago
  A working definition for some apps could be: The user's data should not be available to the system if there isn't an active user session, such that the user's privacy interests are cryptographically protected in event of a breach or data leak occurring when the user is not actively using the system. I wasn't thinking of manual approval of any cryptographic steps. Just that when you log in to work on your data stored in the system, the system can only then decrypt the data, and when you log out, the system forgets the keys until next time. It all depends on the type of app of course.
  - CiPHPerCoder 2y ago
    Okay, this sounds vaguely like a problem that may be solved by "HPKE where the secret key is reconstructed from a threshold secret sharing scheme" (>=2 of N shares needed, 1 held by the service and 1 held by the employee's hardware device, where 1 additional share is held in cold storage for break-glass reasons). I would need to actually sit down and walk through the architecture, threat model, etc. to recommend anything specific. I'm not going to do that on a message board comment, because I probably am missing something.

jnwatson 2y ago

The article has some great points but has an unfortunate title that has lead to an argument the author isn't making. The cryptographic details about preventing confused deputy are excellent and worth studying.

rsync 2y ago

I've read the article and the entire comment thread and nobody is talking about the cloud provider itself ... ? When someone borgs[1] up their data to store at rsync.net we just assume that we are the threat. Of course I don't believe that but it's perfectly rational and we encourage people to think of rsync.net as a threat as they design their backups. Comments in this thread are actually discounting the threat of Amazon personnel, GCS personnel, etc., as if that threat was zero. Not only is it non-zero, I would go further: if you're storing the data on AWS and generating your keys on AWS and managing your keys with AWS ... you're doing it wrong. [1] https://www.borgbackup.org/ https://www.borgbackup.org/

CiPHPerCoder 2y ago
This is an interesting and important point that you raised. > if you're storing the data on AWS and generating your keys on AWS and managing your keys with AWS ... you're doing it wrong. This is a reasonable thing to do if you've decided that you trust AWS and expect any violations of this trust to be dealt with by the legal department. It's less reasonable if you're concerned about AWS employees going rogue and somehow breaking the security of KMS without anyone else knowing. It's even less reasonable to do this if you're concerned about AWS credentials being leaked or compromised, which in turn grants an attacker access to KMS (i.e., a government would be more successful by compelling IAM to grant access than they would trying to subpoena KMS for raw keys). (Sure, you can audit access via CloudTrail, but that's a forensics tool, not a prevention tool.) But that's kind of the point I wrote in the article, no? You need to know your threat model. You've stated yours succinctly, and I think it's a commendable one, but many enterprises are a bit more relaxed.
- jjav 2y ago
  > It's less reasonable if you're concerned about AWS employees going rogue and somehow breaking the security of KMS without anyone else knowing. That's the least of the concerns. Remember AWS is subject to court orders of all types (legitimate ones and NSLs). Even if nobody goes rogue, any data that AWS (or any cloud/SaaS provide) could access, must be assumed to be compromised.

cedws 2y ago

In the context of cloud, it's cargo cult security. Something somewhere says "you must have encryption at rest." I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. So what are you protecting again? If you're storing the decryption keys in KMS in the same cloud, you're not hiding anything from the cloud provider. The only rationale I can think of for doing this is defense-in-depth, but seeing how many companies struggle to even get IAM right I doubt this would help much. Security compliance and real world security are a universe apart. You have encryption at rest with mandated AES-GCM/SHA512? Cool story bro, some teenagers just broke into your network with a bit of social engineering and a 6 year old CVE.

snowstormsun 2y ago
> I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. I find that very easy to belive.
CiPHPerCoder 2y ago
> I find it very hard to believe Amazon's or Google's servers do not already have full disk encryption. I am confident that they do. Even better, they can be configured to use your KMS key rather than the service key, and you can configure KMS to use external key stores (i.e., an HSM in your datacenter outside of AWS's control, that you could theoretically pull the plug on at any time).

wg0 2y ago

On the subject of Encryption ar Rest: On AWS RDS, you can't turn off encryption once you turn it on. The encryption secret keys always stay with AWS and all you can download are the public keys. So I'm not so sure what's the point of encryption at rest in AWS except just to tick off a compliance and regulatory checklist. The private key is with them anyway, just don't encrypt and save few milliwatts of power.

CiPHPerCoder 2y ago
> So I'm not so sure what's the point of encryption at rest in AWS except just to tick off a compliance and regulatory checklist. > The private key is with them anyway, just don't encrypt and save few milliwatts of power. "Them" is Amazon, a company with over 1 million employees, last I checked. It's perfectly reasonable to trust the KMS team to keep your keys secure, even if you don't trust the RDS team to never try to look at your data. I know it's tempting to think of all of AWS as a sort of "Dave" who wears multiple hats, but we're talking about a large company. Protecting against other parts of the same company is still a worthwhile and meaningful security control.
- wg0 2y ago
  As a customer, I don't know neither I do care how they have teamed up internally. Not my problem. From my perspective, the secret keys I don't have. Just AWS has and they can decrypt whatever and whenever they want maybe because they have a warrant or some three letter agency has them do it.
- outworlder 2y ago
  > It's perfectly reasonable to trust the KMS team to keep your keys secure, even if you don't trust the RDS team to never try to look at your data. If the database is live, then the data is able to be decrypted and who knows where it ends up. Encryption at rest solves only the threat scenario where the RDS team has access to the database storage layer. It doesn't do anything to mitigate any threats after it has been read from storage.