3 ms·
For some data structures you get the atomicity for “free” though. For example, a log structured merge tree where writes are done in large transactions could wel
by bjornsing 2y ago
For some data structures you get the atomicity for “free” though. For example, a log structured merge tree where writes are done in large transactions could well do without a WAL and still achieve atomicity.
- nyrikki 2y agoEver have an LSMT quit writing a run half way through? Ever need to try and restore Cassandra where mutation storage order isn't guaranteed, with the last mutation always winning? It is absolutely not "atomic" if you have ever had to do a recovery from SSTable snapshots.
- bjornsing 2y ago> Ever have an LSMT quit writing a run half way through? That should be very easy to “solve” by just writing the length of the SSTable in a header. > Ever need to try and restore Cassandra where mutation storage order isn't guaranteed, with the last mutation always winning? This is more of a distributed systems challenge as I understand it: Cassandra is “multi-master” so can end up in a situation where transaction order is hard to sort out.
- valyala 2y agoA good example of database, which relies on LSM tree properties for protecting against data corruption on unclean shutdown is VictoriaMetrics [1]. [1] https://valyala.medium.com/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704 https://valyala.medium.com/wal-usage-looks-broken-in-modern-...
- amluto 2y agoOne might argue that an LSM tree has a WAL — LSM’s log contains logical records instead of physical changes, but it seems like the essentials are there.
- bjornsing 2y agoYes, I guess that’s another way of phrasing it. But LSM tree implementations often have a separate WAL as well, to ensure durability for small transactions.