4 ms·
The vast majority of companies I've seen consider issuing "SQL DELETE" is sufficient for GDPR compliance. One day, they will be burned when some investigation
by londons_explore 6y ago
The vast majority of companies I've seen consider issuing "SQL DELETE" is sufficient for GDPR compliance.
One day, they will be burned when some investigation finds out that those rows are still sitting in uncompacted tables, or on now-unallocated disk blocks, or on now-remapped SSD sectors.
The only way to be sure the data is deleted is to copy all the data you want to keep to a new drive and burn the old one. Anything less and there is a reasonable chance an expert could recover at least some of the deleted info.
- jacquesm 6y agoAll it takes is one dataleak[+] to blow that wide open. Secure erase is hard. But what is much less hard is to encrypt data and then to limit the problem to getting rid of the decryption key. This reduces the problem in scope to one single datum rather than a whole chain of possible plaintext copies. Secure erase is a contractual requirement for many relationships, it is interesting that none of the major db vendors as far as I know have a secure delete option. There are lots of tricks and attempts to work around it but no official support for such functionality afaik. And that's before we get into VM snapshots, database snapshots, backup copies, copies that were made by developers or data scientists for test purposes (of course, nobody ever does that) and so on. Nasty little problem. [+] or some employee shooting their mouth off in an online forum.
- xwdv 6y agoGod help you if your decryption key is destroyed somehow and you no longer have access to your own data. Fuck that.
- jacquesm 6y agoIf you can't keep your keys around in a safe way you shouldn't be in this business to begin with.
- mschuster91 6y ago> But what is much less hard is to encrypt data and then to limit the problem to getting rid of the decryption key. This reduces the problem in scope to one single datum rather than a whole chain of possible plaintext copies. Key issue: you can't do any operations on encrypted data, essentially you're killing off your database. Homomorphic encryption is academic research, not something that is widely available and supported in common open and closed source databases. The best you can get is a database that encrypts the on disk data (Oracle TDE), but that only protects against a server being stolen or hacked on the OS level.
- jacquesm 6y agoI fully expect that we will see application level and system level tools that are compliant with the law pop up any day now, the need is certainly there. Also, for purposes of operations on data it all depends on what the column holds. For instance, you don't actually need access to fields such as names and dates of birth if you have a client ID and that is yours, not the customers and so you could leave that field in plain text. Any operation would then need that client ID but that's workable. You could even say that if you would need that decryption key for anything other than user or controller directed computation that you are probably doing something you shouldn't be doing. In all other cases the context is clear, consent has been obtained and the data can be decrypted if required.
- jandrewrogers 6y agoSecure delete is not implemented in databases because it is extraordinarily expensive, and destroys the performance of all other non-delete operations. When you say “encrypt data”, you are ignoring the fact that it can’t be implemented as “same data structures, just encrypted”. Encryption puts constraints on data structures and data representation that are fundamentally incompatible with and adverse to the design and functioning of most databases in existence, even ignoring the other ugly operational issues with that approach which no one thinks about. It would require radically redesigning the internals of most database engines, which is not really a practical option. I’ve thought a lot about what it would take to design a “delete optimized” database kernel every since GDPR became a thing. In principle it is possible, but I seriously doubt anyone would use a database that is literally orders of magnitude slower and less scalable for everything except delete operations. Would it be acceptable to increase the resource intensity of databases by 10x (and environmental footprint implied) to get “real” deletes? That is the tradeoff here. This has precedent in SQL databases designed for high-assurance applications with ultra-fine access and visibility controls. When the average software engineer understands how they work, it sounds like a great idea for having more secure data and they wonder why it doesn’t seem to exist. The reality is that they do exist but they are so abysmally slow for even elementary things that no one would ever dream of using it unless there is a narrow government requirement.
- jacquesm 6y agoThe law doesn't really care about your notion of what can be done effeciently, if it can be done then you should probably do it or risk being found in violation of the law. There is some very specific language to that effect in the GDPR to make sure that it is clear that 'whatever you could reasonably do' needs to be done. You can then go and argue that you felt it wasn't reasonable but I doubt that will fly. But you're totally right that this is a tricky problem and hard to do properly. On the plus side, field level encryption is something that we've been doing for ages and that already works quite well. If you design carefully you can even get some processing done on those fields though it will take some major breakthroughs in DB engine design before you can have your cake and eat it too, in the sense that you can be both legally compliant and do all the kinds of processing you can do today. I even doubt if that is desirable, lots of those examples of processing should probably not be done in the first place.
- icedchai 6y agoThey probably know the odds of an "investigation" actually occurring is very low. The odds of one occurring with that level, or probably any level, of technical sophistication is near zero.
- jacquesm 6y agoFrom regulators: yes, they typically work reactive, until there is some kind of breach you'll never hear from them. From investors, customers and potential acquirers audits are pretty common and becoming more so every day.