4 ms·

>>It turns out that append-only databases are well-suited for HR records It may seem so, bu then You find out what are Your governments limit's for storing wor

by v-erne 4y ago

>>It turns out that append-only databases are well-suited for HR records It may seem so, bu then You find out what are Your governments limit's for storing workers data, and Your company lawyers forces You to make it possible to delete everything that past that limit to limit legal risk and be gdpr compliant. And at this moment You find out that world is not constant and far from Your ideal model of spherical cow.

function_seven 4y ago
Can we design an append-only database that also allows for deletions if you replace those deletions with a direct reference to the hash that was once computed at that point in the chain? Yes, I know this isn’t strictly “append only”, but it still gives the ability to prevent silent updates and silent deletes. Any record that’s deleted must be replaced with something that indicates that it was deleted.
- Drakim 4y ago
  If the database is distributed you have no way of enforcing that the other peers don't keep shadow copies of the data that is supposed to be deleted. If you know you can trust all your peers to act in good faith, then you might as well just use a regular database as none of your peers will edit old records due to being good actors.
  - function_seven 4y ago
    Yeah, what I'm describing is more like a journaled database, not bLoCkChaIn tEcHnOloGy. There's a chain for sure, but nothing that allows it to be decentralized, or 100% immutable, etc.
- inkyoto 4y ago
  Such a database already exists, and it is called AWS QLDB. Deleting a document revision in QLDB does not actually delete it, but rather moves the last known revision into a shadow history table that accompanies every table in QLDB. The full document change history lookup has to accommodate in the code path a fallback to the history table when a document revision can't be located in the main data entity table. If no document revision exists in either table, it has never existed in the database.
- some_furry 4y ago
  Use envelope encryption and wipe the key after the fact. Bonus: use a committing AEAD mode and use the previous record as AEAD when writing the record. You can keep the ciphertext in the ledger forever. Zeroing out the wrapped key makes it unretrievable.
oblio 4y ago
Small nitpick: "You" and "Your" don't need to be capitalized. It makes reading the text a bit weird.
- v-erne 4y ago
  Thank you [see what I did here :)] for pointing this out - I'm not native and on top of that used to learn german (never went anywhere with it, but the capitalization of every noun somehow stayed with me longer than necessary). Add english oddities such as capitalizing first person pronoun (which I do, as you probably noticed) and it's kinda easy to be oblivious that I do this.

by v-erne 4y ago

function_seven 4y ago

Can we design an append-only database that also allows for deletions if you replace those deletions with a direct reference to the hash that was once computed at that point in the chain? Yes, I know this isn’t strictly “append only”, but it still gives the ability to prevent silent updates and silent deletes. Any record that’s deleted must be replaced with something that indicates that it was deleted.

Drakim 4y ago
If the database is distributed you have no way of enforcing that the other peers don't keep shadow copies of the data that is supposed to be deleted. If you know you can trust all your peers to act in good faith, then you might as well just use a regular database as none of your peers will edit old records due to being good actors.
- function_seven 4y ago
  Yeah, what I'm describing is more like a journaled database, not bLoCkChaIn tEcHnOloGy. There's a chain for sure, but nothing that allows it to be decentralized, or 100% immutable, etc.
inkyoto 4y ago
Such a database already exists, and it is called AWS QLDB. Deleting a document revision in QLDB does not actually delete it, but rather moves the last known revision into a shadow history table that accompanies every table in QLDB. The full document change history lookup has to accommodate in the code path a fallback to the history table when a document revision can't be located in the main data entity table. If no document revision exists in either table, it has never existed in the database.
some_furry 4y ago
Use envelope encryption and wipe the key after the fact. Bonus: use a committing AEAD mode and use the previous record as AEAD when writing the record. You can keep the ciphertext in the ledger forever. Zeroing out the wrapped key makes it unretrievable.

oblio 4y ago

Small nitpick: "You" and "Your" don't need to be capitalized. It makes reading the text a bit weird.

v-erne 4y ago
Thank you [see what I did here :)] for pointing this out - I'm not native and on top of that used to learn german (never went anywhere with it, but the capitalization of every noun somehow stayed with me longer than necessary). Add english oddities such as capitalizing first person pronoun (which I do, as you probably noticed) and it's kinda easy to be oblivious that I do this.