4 ms·

"Art. 17 GDPR: Right to erasure ('right to be forgotten') 1. The data subject shall have the right to obtain from the controller the erasure of personal data co

"Art. 17 GDPR: Right to erasure ('right to be forgotten') 1. The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay..." How does this right affect a website backed by an immutable database? Is it enough for data to be superseded by later data, such as an assertion that the prior data is defunct? Or does it have to be actually erased? Can it be considered erased for Article 17 purposes if the database owner can still access it? Is a website with an immutable database illegal under the GDPR after a court order to delete something has been received? Datomic devs want to know. https://gdpr-info.eu/art-17-gdpr/ https://gdpr-info.eu/art-17-gdpr/

toolslive 6y ago
you could encrypt every record (with its individual nonce), and do the merkle tree has over the encrypted record. To erase the contents of the record, you just have to forget the encryption key. A variation of this trick is used in block devices that offer a fast wipe. Instead of having to fill every block with a bit pattern to wipe the contents, they just have to change the current encryption key. Once that's done the device is effectively filled with random bits.
- specialist 6y ago
  Exactly. Just like a password wallet. Store the hash of passwords, not the actual passwords. Then disallow password resets. Then you effectively "forget" your account if you lose your password. The book Translucent Databases (2nd ed) [2009] explains clever strategies for applying this technique to protect sensitive data. It's brilliant. https://www.wayner.org/node/39 https://www.wayner.org/node/39 https://www.amazon.com/gp/product/1441421343 https://www.amazon.com/gp/product/1441421343 Meta: I remain disappointed by the obscurity of this book and translucent techniques. A long time friend recently asked me about GDPR compliance and so forth, in prep for reworking stuff to allow proper audits. Very tech savvy. The translucent notions just could not compute. So their efforts went down the conventional rabbit hole of actually deleting data. Which I don't consider practical or auditable. How can you be sure an org deleted every record, log, backup, etc? You can't.
  - toolslive 6y ago
    encryption the records. fine, but how do you handle the indexing of the attributes?
    - specialist 6y ago
      I think I understand your question. "Translucent" was coined to describe somewhere between all cleartext (transparent) and all encrypted (opaque). Generally, only sensitive fields are encrypted, so no record level encryption. The cleartext fields can still be searched, indexed. The book shows many clever schemas for mixing and matching hashes, keys, and cleartext. I used translucent strategies for storing patient data. It's pretty straightforward after some experience. Though I admit I didn't have any resources to review, audit our efforts.
szemet 6y ago
IANAL but I recall some vauge memories that Datomic was to be modifed, to provide real physical deletion according to the GDPR reqs...
- slifin 6y ago
  I think you can https://docs.datomic.com/on-prem/excision.html https://docs.datomic.com/on-prem/excision.html on prem you can choose not to store history with :db/noHistory per fact then just update the record in place Datomic cloud though I'm not sure
  - 6gvONxR4sf7o 6y ago
    An in-place update seems counter to the whole immutable premise. There must be a middle ground of in-place with record. Like, (1/2/2020, John, oldAddress) becomes (1/2/2020, John, oldAddress) --> (3/4/2020, John, newAddress) becomes (1/2/2020, deletionRequestName1, deletionRequestAddress1) --> (3/4/2020, deletionRequestName1, deletionRequestAddress2) --> (5/6/2020, GDPRDelete, deletionRequestName1, deletionRequestAddress1, deletionRequestAddress2) or something. That doesn't feel quite right, but it feels like there's some nice combo somewhere that keeps the auditing and history and most of the simplicity. Or maybe there's a cryptographic solution, like (1/2/2020, ALKXJDS, CNDSLKDJ) becomes (1/2/2020, ALKXJDS, CNDSLKDJ) --> (3/4/2020, ALKXJDS, QWERTYYUIOP) becomes (1/2/2020, ALKXJDS, CNDSLKDJ) --> (3/4/2020, ALKXJDS, QWERTYYUIOP) --> (5/6/2020, we threw away the cryptographic keys to decode ALKXJDS's info).
    - derefr 6y ago
      If it's anything like CouchDB, there's a compaction sweep that normally puts small history chunks together into larger history chunks, but which also has the responsibility of dropping any history before a tombstone record on the floor.
      - ashtonkem 6y ago
        Datomic stores its data in a list of “facts”. Normally these facts are append only, but excision is a special case. Think of it this way: normally a database entry is represented as a row. You could also represent this exact same data as a list of triples: primary key, attribute name, and attribute value. Same data, different representation. Datomic stores its data in a series of quadruples: primary key, attribute name, attribute value, and transaction. These facts are append only; when data is normally deleted the transaction includes the fact that the data is being deleted, not added. Under the hood Datomic processes these facts to produce the current state of the world, but the old data is there if you ask for it. Excision deletes facts from the database, which both violates a lot of assumptions about how the database works, and permanently removes the data. In the process of excision it leaves a single record to indicate that something was removed, without clarifying what. It’s something they only recommend for regulatory compliance, as it eliminates a lot of the value in the database.
jasonwatkinspdx 6y ago
Immutable doesn't necessarily mean that it's literally impossible to rewrite history or redact history. It's more a change in the default of how things are done. Some merkle tree variants directly support deletions. You keep around the hash of the deleted subtree, so the overall structure still has a verifiable hash history, but you lose the data that generated the hash. Another simple approach is to use per user encryption keys, and when a user wants deletion just nuke their key in the system that stores keys.
smitty1e 6y ago
My understanding from reading the O'Reilly book on the subject is that the blockchain is 'just' cryptographic hashes, not the data proper.
vbezhenar 6y ago
What does it mean "actually erased"? If I delete file, its contents are still on disk. If SSD remaps block, it might be inaccessible by operating system at all but it still technically contains data.
- derefr 6y ago
  The spirit of "right to be forgotten" is a right to not have past events unduly influence your public image. I would assume that—unless it's been proven that you can GDPR-takedown-request a credit agency to erase your credit history—there's no "right to be forgotten" as applies to never-publicized, company-internal data about you.
  - ealexhudson 6y ago
    That's not quite it. The different legal bases for processing personal data come with different rights; one of them is the right of erasure. It doesn't matter if the data is public or private. This isn't a general bazooka - you'd be unlikely to erase the data that generates a credit score, for example, but then there will be equivalent rights (right to explanation of an automated decision, plus the right to correct a mistaken record). All of these rest on the ability of the subject to gain access to their record in the first place too. Immutability isn't an automatic problem for GDPR - you're allowed to take backups of databases too! - but it is axiomatically more difficult to be in compliance with such an arrangement.
ashtonkem 6y ago
Other immutable databases (Datomic) provide the ability to permanently erase entities from the database to meet regulatory needs. It leaves behind a single record to indicate that something was excised from the database.

toolslive 6y ago

you could encrypt every record (with its individual nonce), and do the merkle tree has over the encrypted record. To erase the contents of the record, you just have to forget the encryption key. A variation of this trick is used in block devices that offer a fast wipe. Instead of having to fill every block with a bit pattern to wipe the contents, they just have to change the current encryption key. Once that's done the device is effectively filled with random bits.

specialist 6y ago
Exactly. Just like a password wallet. Store the hash of passwords, not the actual passwords. Then disallow password resets. Then you effectively "forget" your account if you lose your password. The book Translucent Databases (2nd ed) [2009] explains clever strategies for applying this technique to protect sensitive data. It's brilliant. https://www.wayner.org/node/39 https://www.wayner.org/node/39 https://www.amazon.com/gp/product/1441421343 https://www.amazon.com/gp/product/1441421343 Meta: I remain disappointed by the obscurity of this book and translucent techniques. A long time friend recently asked me about GDPR compliance and so forth, in prep for reworking stuff to allow proper audits. Very tech savvy. The translucent notions just could not compute. So their efforts went down the conventional rabbit hole of actually deleting data. Which I don't consider practical or auditable. How can you be sure an org deleted every record, log, backup, etc? You can't.
- toolslive 6y ago
  encryption the records. fine, but how do you handle the indexing of the attributes?
  - specialist 6y ago
    I think I understand your question. "Translucent" was coined to describe somewhere between all cleartext (transparent) and all encrypted (opaque). Generally, only sensitive fields are encrypted, so no record level encryption. The cleartext fields can still be searched, indexed. The book shows many clever schemas for mixing and matching hashes, keys, and cleartext. I used translucent strategies for storing patient data. It's pretty straightforward after some experience. Though I admit I didn't have any resources to review, audit our efforts.

szemet 6y ago

IANAL but I recall some vauge memories that Datomic was to be modifed, to provide real physical deletion according to the GDPR reqs...

slifin 6y ago
I think you can https://docs.datomic.com/on-prem/excision.html https://docs.datomic.com/on-prem/excision.html on prem you can choose not to store history with :db/noHistory per fact then just update the record in place Datomic cloud though I'm not sure
- 6gvONxR4sf7o 6y ago
  An in-place update seems counter to the whole immutable premise. There must be a middle ground of in-place with record. Like, (1/2/2020, John, oldAddress) becomes (1/2/2020, John, oldAddress) --> (3/4/2020, John, newAddress) becomes (1/2/2020, deletionRequestName1, deletionRequestAddress1) --> (3/4/2020, deletionRequestName1, deletionRequestAddress2) --> (5/6/2020, GDPRDelete, deletionRequestName1, deletionRequestAddress1, deletionRequestAddress2) or something. That doesn't feel quite right, but it feels like there's some nice combo somewhere that keeps the auditing and history and most of the simplicity. Or maybe there's a cryptographic solution, like (1/2/2020, ALKXJDS, CNDSLKDJ) becomes (1/2/2020, ALKXJDS, CNDSLKDJ) --> (3/4/2020, ALKXJDS, QWERTYYUIOP) becomes (1/2/2020, ALKXJDS, CNDSLKDJ) --> (3/4/2020, ALKXJDS, QWERTYYUIOP) --> (5/6/2020, we threw away the cryptographic keys to decode ALKXJDS's info).
  - derefr 6y ago
    If it's anything like CouchDB, there's a compaction sweep that normally puts small history chunks together into larger history chunks, but which also has the responsibility of dropping any history before a tombstone record on the floor.
    - ashtonkem 6y ago
      Datomic stores its data in a list of “facts”. Normally these facts are append only, but excision is a special case. Think of it this way: normally a database entry is represented as a row. You could also represent this exact same data as a list of triples: primary key, attribute name, and attribute value. Same data, different representation. Datomic stores its data in a series of quadruples: primary key, attribute name, attribute value, and transaction. These facts are append only; when data is normally deleted the transaction includes the fact that the data is being deleted, not added. Under the hood Datomic processes these facts to produce the current state of the world, but the old data is there if you ask for it. Excision deletes facts from the database, which both violates a lot of assumptions about how the database works, and permanently removes the data. In the process of excision it leaves a single record to indicate that something was removed, without clarifying what. It’s something they only recommend for regulatory compliance, as it eliminates a lot of the value in the database.

jasonwatkinspdx 6y ago

Immutable doesn't necessarily mean that it's literally impossible to rewrite history or redact history. It's more a change in the default of how things are done. Some merkle tree variants directly support deletions. You keep around the hash of the deleted subtree, so the overall structure still has a verifiable hash history, but you lose the data that generated the hash. Another simple approach is to use per user encryption keys, and when a user wants deletion just nuke their key in the system that stores keys.

smitty1e 6y ago

My understanding from reading the O'Reilly book on the subject is that the blockchain is 'just' cryptographic hashes, not the data proper.

vbezhenar 6y ago

What does it mean "actually erased"? If I delete file, its contents are still on disk. If SSD remaps block, it might be inaccessible by operating system at all but it still technically contains data.

derefr 6y ago
The spirit of "right to be forgotten" is a right to not have past events unduly influence your public image. I would assume that—unless it's been proven that you can GDPR-takedown-request a credit agency to erase your credit history—there's no "right to be forgotten" as applies to never-publicized, company-internal data about you.
- ealexhudson 6y ago
  That's not quite it. The different legal bases for processing personal data come with different rights; one of them is the right of erasure. It doesn't matter if the data is public or private. This isn't a general bazooka - you'd be unlikely to erase the data that generates a credit score, for example, but then there will be equivalent rights (right to explanation of an automated decision, plus the right to correct a mistaken record). All of these rest on the ability of the subject to gain access to their record in the first place too. Immutability isn't an automatic problem for GDPR - you're allowed to take backups of databases too! - but it is axiomatically more difficult to be in compliance with such an arrangement.

ashtonkem 6y ago

Other immutable databases (Datomic) provide the ability to permanently erase entities from the database to meet regulatory needs. It leaves behind a single record to indicate that something was excised from the database.