7 ms·

I only skimmed that post, and havent dealt with mysql in years, but .... that bit has another wiggle with writes committed to one of many replicas. Im presuming

I only skimmed that post, and havent dealt with mysql in years, but .... that bit has another wiggle with writes committed to one of many replicas. Im presuming thats any one of many. Which means I can now have writes committed not just in two places (terrible), but to N number of places (zomg). Recovering that (linearly) would tend towards impossible without something like a vector clock. But if you had a reliable clock of changes you probably wouldnt be in that mess at all.

teraflop 8y ago
You don't need vector clocks for master/slave replication, even if there's 1 master and N slaves. For consistency, all of the slaves should be applying the stream of updates to their local replicas in the same order that they happened on the master. That means each slave has seen a prefix of the master's data; its state can be uniquely represented by a position in the replication log. So you can easily determine which replica is most up-to-date by taking the one with the largest position. (If you're doing synchronous replication correctly, at least one replica is guaranteed to be completely up-to-date, even after an unexpected failure.) Vector clocks only come into play when you are simultaneously accepting writes on multiple masters, and you need to keep track of which updates have already been applied from each source.
- donavanm 8y ago
  > If you're doing synchronous replication correctly, at least one replica is guaranteed to be completely up-to-date, even after an unexpected failure. Exactly? Maybe? Im reading a lot in to little information here. It wasnt clear to me if "one or more" was the master + a single specific replica, or something closer to master + 1/N. And then there's the automated optimistic leader election. Combine those with (multiple) partitions and you could have a very bad day. Rereading their RCA now I'm guessing the above isnt the case. "Restore the entire datacenter" reads more like they preserved the ~40 seconds of east coast partition writes, restored everything in east to a known checkpoint, and resumed replication off the more advanced masters in the west coast. If it's "just" two masters per cluster, both of which are available and internally consistent, you "just" have to reconcile those two write streams.

teraflop 8y ago

You don't need vector clocks for master/slave replication, even if there's 1 master and N slaves. For consistency, all of the slaves should be applying the stream of updates to their local replicas in the same order that they happened on the master. That means each slave has seen a prefix of the master's data; its state can be uniquely represented by a position in the replication log. So you can easily determine which replica is most up-to-date by taking the one with the largest position. (If you're doing synchronous replication correctly, at least one replica is guaranteed to be completely up-to-date, even after an unexpected failure.) Vector clocks only come into play when you are simultaneously accepting writes on multiple masters, and you need to keep track of which updates have already been applied from each source.

donavanm 8y ago
> If you're doing synchronous replication correctly, at least one replica is guaranteed to be completely up-to-date, even after an unexpected failure. Exactly? Maybe? Im reading a lot in to little information here. It wasnt clear to me if "one or more" was the master + a single specific replica, or something closer to master + 1/N. And then there's the automated optimistic leader election. Combine those with (multiple) partitions and you could have a very bad day. Rereading their RCA now I'm guessing the above isnt the case. "Restore the entire datacenter" reads more like they preserved the ~40 seconds of east coast partition writes, restored everything in east to a known checkpoint, and resumed replication off the more advanced masters in the west coast. If it's "just" two masters per cluster, both of which are available and internally consistent, you "just" have to reconcile those two write streams.