4 ms·
People need to stop writing stuff about distributed systems if they won’t explicitly talk about the P in CAP theorem. What happens here in the event of a netwo
by ianamartin 7y ago
People need to stop writing stuff about distributed systems if they won’t explicitly talk about the P in CAP theorem.
What happens here in the event of a network partition? There are several failure modes possible if I’m reading the diagram right. Network failure between the Aurora node and the shared storage layer.
I would expect that to result in a failed transaction and rollback, right? That’s not really all that available though, is it?
What about a network partition between availability zones? Maybe your local thinks it has a quorum because it can no longer see a different AZ? What happens when the partition is healed? How do the nodes reconcile? What if you’re working with multiple AZs and you have application nodes in each AZ writing to a cluster of nodes that all think they have a quorum because they can’t see each other?
What happens to those writes after the partition? Is every transaction terminated rolled back because us-east-1 can’t see us-west-3?
If they aren’t, how do the nodes achieve consensus? Are we doing a mongoDB thing here and just tossing everything after the last shared state? And even if that does work out,
What’s going on? And how is it achieved.
Look, if you need transactions at all, you most likely need isolation level: serializable. And that’s incredibly hard to get right.
Many systems really don’t need any of that at all, but when you need it, it has to be right.
This doesn’t address any of the really hardest parts of distributed systems. It doesn’t address why I would take any application at all that stores data in a RDBMS and use this.
I’m sure everyone there on that team is working hard. But people have to stop writing about distributed systems like this.
- redis_mlc 7y agoThe above comment is, well, not very informed. CAP is a simple, generic academic model. Researchers and implementers moved on from that a long time ago. In the case of Google, they have partition-free networks for newer databases, so CAP rarely applies. Galera/Percona Cluster has solutions for all the above also. Generally multi-master writes continue, and the behind server either catches up or gets automatically rebuilt. MySQL 8 has something equivalent to Galera. I would recommend testing (or waiting) a year for multi-master Aurora to bake. In the meantime, you can use Percona Cluster if you can't wait. You can learn more here about the state-of-the-art in multi-region, multi-master RDBMS' here: http://www.jebriggs.com/blog/2017/11/internet-latency-and-multi-master-database-transactions/ http://www.jebriggs.com/blog/2017/11/internet-latency-and-mu... Source: DBA who's operated Percona Cluster for 5+ years. Contact me if you need extreme database performance or availability.
- kjeetgill 7y agoI don't strictly disagree with you, but the parent has a point. It's not a technical outline of the difficult tradeoffs distributed systems must make[0], rather an ad copy where they can promise you the world and it's hard to tell what's exactly true. Even igoring Amazon, MySQL/Percona's mha story has had a history of buggy, weak, imprecise promises. [0] I guess Google level special sauce gets a pass since they have the most published about specially clocked hardware, and how it changes DB design.
- zzzcpan 7y ago> In the case of Google, they have partition-free networks for newer databases, so CAP rarely applies. No, asynchronous networks cannot physically be partition-free. They are actually always partitioned. > Galera/Percona Cluster has solutions for all the above also They don't have solutions to all of the above.
- ianamartin 7y agoThis is an incredible comment. You have some real chutzpah, sir. I respect that. You are as wrong as can possibly be, but this is a bold move.
- SahAssar 7y agoWhat exactly is a "partition-free network"? In my understanding no such thing exists, a partition can happen even if all the servers run on the same physical hardware (because of networking stacks, bugs, misconfiguration and so on), and even if they think that no such thing can happen if runs for eternity it will eventually happen. The question that CAP asks is what happens when a partition happens? Does the DB C: Stay consistent but not available A: Stay available but not consistent There is no other option.
- zzzcpan 7y agoAmazon is likely the only one who can do this well today and they explain consistency in the post.
- ianamartin 7y agoI don't think Consistency was explained in the post. But that may be a weakness on my part.