4 ms·
A concrete example we've faced. A certain operation requires writing data to N flaky services. You successfully write to N-1 of them, but the Nth fails. Now wha
by msluyter 12y ago
A concrete example we've faced. A certain operation requires writing data to N flaky services. You successfully write to N-1 of them, but the Nth fails. Now what do you do?
If these N things were just database writes to the same DB, transactions would save you, as you could just rollback. Without that, the answer has to be handled in code -- do you reverse the previous changes (if possible) by sending delete events, or leave the system in some sort of half-baked state and rectify things later via some other process? (I'm interested in hearing of other options...)
- MichaelGG 12y agoOn my phone so boo cite. Look up Amazon apologetic computing. They also have a presentation describing their general distributed approach. The main thing is that you have a bunch of components that you expect to fail. If they don't get confirmation, they send another message to the next component. Then, there's a stateless, unambiguous way to determine which message "wins" in case of conflict. For instance, an order submits two shipping requests for the same order. They use, for instance, the highest message ID (a GUID) to avoid duplicate processing. It's all very neat, though I'm doing a bad job explaining. Search for Amazon architectural presentations and hopefully you can find it. It may have been on James Hamilton's blog, maybe.
- snowwolf 12y agoYou are basically describing the consensus problem [1] and one of the most popular algorithms used to solve is the paxos algorithm [2] [1] - http://en.wikipedia.org/wiki/Consensus_%28computer_science%29 http://en.wikipedia.org/wiki/Consensus_%28computer_science%2... [2]- http://en.wikipedia.org/wiki/Paxos_(computer_science) http://en.wikipedia.org/wiki/Paxos_(computer_science)