3 ms·
I am a huge fan of append-only logs as a fundamental architectural principle. The Log [1] should be required reading for any CS undergraduate. [1]: https://eng
by hiAndrewQuinn 2y ago
I am a huge fan of append-only logs as a fundamental architectural principle. The Log [1] should be required reading for any CS undergraduate.
[1]: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying https://engineering.linkedin.com/distributed-systems/log-wha...
- kasey_junk 2y agoI love them so much that I’ve noodled with building a programming language optimized for using them. Things like types that encode what events are legal in a log, first class support for data versions, fast file read and writes, etc
- daxfohl 2y agoHow do you do GDPR takedowns?
- jaseemabid 2y agoImmutable append only persistent log doesn't imply store everything _forever_. If you want to remove something you could add a tombstone record (like Cassandra) and eventually remove the original entry during routine maintenance operations like repacking into a more efficient format, archival into cold storage, TTL handling etc.
- UltraSane 2y agoBest method is to encrypt data with a per user key so you if you delete the encryption key that customer's data cannot be read. You can create a new customer key at a fixed time interval to make deletion more granular.