3 ms·
Every time the topic of consistency in distributed systems arise I see this idea mentioned in a way or the other that the problem is how hard strong consistency
by antirez 11y ago
Every time the topic of consistency in distributed systems arise I see this idea mentioned in a way or the other that the problem is how hard strong consistency is. To be honest is a problem solved a long time ago by consensus algorithms: there is a lot more work to do in the future in order to explore other useful consistency models, but if you want strong consistency you know you have "just" to use a consensus algorithm like Paxos or Raft or Viewstamped replication or whatever, and you end with a CP system. Of course it is non trivial even to implement those systems correctly, but the point is, most implementations tested by Aphyr don't attempt to do this at all most of the times.
There will be bugs but in the end if your goal is to write a consistent store, you can do it. Bugs in wanna-be CP systems and/or failed attempts at implementing CP because of home-made protocols with flaws are part of the issue and part of what Aphyr's research uncovers, but IMHO the central point is a different one: CP systems have performance limits so many real world systems don't even try to be CP, which is fine, it's up to the designer to pick the DB design and tradeoffs, but you need to document it properly. What happens during partitions? What consistency model the system employs if it does not feature strong consistency? Is the system Available during partitions? And so forth. As long as the documentation is honest, it's up to the user to understand, for its use case, if the system is a good fit or not.
What we are seeing often is an attempt to informally document what should be well specified. Sometimes there are also incorrect things stated in bold letters in the documentation, like "the system is consistent but sacrifices partition tolerance" which does not mean anything in the context of partitions actually happening in the real world.
Aphyr's effort and our collective effort as people working, implementing and using databases, in my opinion, is not to end with databases designed all the same way or all providing strong guarantees, but with clear understandable and honest documentation describing the system behavior.
- baudehlo 11y agoThe only thing I'd take out of your comment is "all providing strong guarantees". I'd say that if you document that it does provide those guarantees that it damn well better do that (modulo bugs of course).
- rdtsc 11y ago> but IMHO the central point is a different one: CP systems have performance limits so many real world systems don't even try to be CP They try on paper, so to speak. They might have a CP component, but it is not in the datapath -- it is there for maybe cluster configuration, or say to elect a leader. And then they can claim they have CP because everything goes through the leader, and the leader was elected by a CP component. However without realizing they just pushed all the error cases into corners -- when the membership changes (nodes die, die in unpredictable ways, become partitioned, new nodes added...). They never test that extensively, which is a hard thing to do, because there are so many cases and combinations. The bottom line is even if they have a Paxos or Raft component in their system doesn't mean their data is stored consistently.