3 ms·
Log structured merge is the last piece of your architecture. It solves the latency part. It can also solve your durability issues, if you fsync when necessary a
by mmalone 8y ago
Log structured merge is the last piece of your architecture. It solves the latency part. It can also solve your durability issues, if you fsync when necessary and you understand your hardware.
- a1369209993 8y agoYeah, sorry, I assumed log/journal plus reapply-on-recovery went without saying. You can't have guaranteed durability (short of blockchain-style distributed stores) because it's always possible for your database server to suffer a unrecoverable hardware failure between transmitting "transaction success" and taping out its weekly backup. If all else success, your datacenter catching fire and burning down would suffice. So it's only ever a matter of reducing (not eliminating) the probability of having to roll back commited transactions. (Tldr: fsync can't help if your hard drive melted.)
- mmalone 8y agoOh, I assumed we were talking distributed. Fsync to more than one disk. My biases :).
- a1369209993 8y agoFsync to more than one disk is great way to reduce the probability of forced rollback, but simutaneous failure is a still possible. Then again, ACI does assume you aren't having CPU/memory/etc failures, so to the extent that you count anything as guaranteed, it is probably possible to 'guarantee' durablity. My original point was more that "minimally adequate" isn't that hard compared to going from there to a high performance, general purpose system (and also a offhand snipe at SQL).