6 ms·

How does it handle the possible inconsistency due to the asynchronous replication?

by amock 15y ago

fleitz 15y ago
Write Ahead Logging. Async replication doesn't produce inconsistency it produces uncommitted transactions. Every database produces them when it goes down whether it replicates or not. When the master comes up all it has to do is reverse the transactions that the slave didn't receive. Voila, consistent database.
- amock 15y ago
  What about inconsistencies with data outside the database? Things like credit card transactions or other external API calls that were recorded on the master but not the slave will be inconsistent with your slaves view of the world. Is there a standard way of dealing with those kind of things or is that usually handled manually?
  - fleitz 15y ago
    You use the logs from the slave to commit the 2nd portion of a two-phase commit, and immediately stop processing new transactions when you only have the slave up. Your hypothetical API does support two-phase commit correct? Because if it doesn't you have lots of solutions for losing data/creating inconsistent data anyway.
byoung2 15y ago
How does it handle the possible inconsistency due to the asynchronous replication? That would be a big question for me. I can sync databases easily and instantaneously within an availability zone, and nearly instantaneously across availability zones within a region. But once I have to replicate across regions, I add latency to the mix. If replication from US-East to US-West is asynchronous, how would I reconcile the two? I suppose with a catastrophic failure of the primary database in US-East, I could just write off any data that wasn't replicated before the failure, but that doesn't seem like a solid solution. Would it be better to write a transaction layer into the app, so that data isn't considered committed until it has been written and replicated across multiple regions?
- rosser 15y ago
  There are all kinds of solutions available to you in that scenario. Two-phase commit, on DBs that support it, would probably go a long way towards enabling a transaction layer like you describe. Usually, though, discussions of DR should start with determining what kinds of RPO and RTO you're willing to pay for and then evaluating which of the available solutions will get you there.

How does it handle the possible inconsistency due to the asynchronous replication?

by amock 15y ago

How does it handle the possible inconsistency due to the asynchronous replication?

Write Ahead Logging. Async replication doesn't produce inconsistency it produces uncommitted transactions. Every database produces them when it goes down whether it replicates or not. When the master comes up all it has to do is reverse the transactions that the slave didn't receive. Voila, consistent database.

amock 15y ago
What about inconsistencies with data outside the database? Things like credit card transactions or other external API calls that were recorded on the master but not the slave will be inconsistent with your slaves view of the world. Is there a standard way of dealing with those kind of things or is that usually handled manually?
- fleitz 15y ago
  You use the logs from the slave to commit the 2nd portion of a two-phase commit, and immediately stop processing new transactions when you only have the slave up. Your hypothetical API does support two-phase commit correct? Because if it doesn't you have lots of solutions for losing data/creating inconsistent data anyway.

byoung2 15y ago

How does it handle the possible inconsistency due to the asynchronous replication? That would be a big question for me. I can sync databases easily and instantaneously within an availability zone, and nearly instantaneously across availability zones within a region. But once I have to replicate across regions, I add latency to the mix. If replication from US-East to US-West is asynchronous, how would I reconcile the two? I suppose with a catastrophic failure of the primary database in US-East, I could just write off any data that wasn't replicated before the failure, but that doesn't seem like a solid solution. Would it be better to write a transaction layer into the app, so that data isn't considered committed until it has been written and replicated across multiple regions?

rosser 15y ago
There are all kinds of solutions available to you in that scenario. Two-phase commit, on DBs that support it, would probably go a long way towards enabling a transaction layer like you describe. Usually, though, discussions of DR should start with determining what kinds of RPO and RTO you're willing to pay for and then evaluating which of the available solutions will get you there.