3 ms·

> You can make partition tolerance arbitrarily low I guess you mean 'the need for partition tolerance'. And that is honestly a better way of reading the P - ef

> You can make partition tolerance arbitrarily low I guess you mean 'the need for partition tolerance'. And that is honestly a better way of reading the P - effectively as 'partitionable'. CAP says if you are making a partitionable system, it can't guarantee both consistency and availability. If you want consistency and availability, you will have to make the system unpartitionable - effectively, not distributed. The idea of 'partition tolerance' just makes no sense, because it's not really possible to make a 'partition intolerant' partitionable system. If the definition of a 'partition tolerant' system is one which actively determines, in the face of network partition, whether to stop answering requests because it can't communicate with other nodes, or to nonetheless answer requests in the knowledge that its answers might be inconsistent... ... how would a 'partition intolerant' node be able to do anything different? I suspect the most likely outcome for a system which can't tolerate partition is to lose both consistency AND availability.

derefr 5y ago
> 'partition intolerant' partitionable system How about: any time there's a network partition, each partition decides on a new identity for itself, and becomes a new cluster that will never join back into the original, larger cluster again. People pick sides and never interact with people on the other side again. Consistency! (Because every node in each new smaller sub-cluster is now consistent with any other node said nodes will ever interact with again.) Availability! (Because all the nodes stay online.) Partition tolerance—in the sense of the partition ever being resolved? Nope. I'm not just playing around, either; I'm describing non-unilaterally-assented hard forks in blockchains — e.g. the split between Ethereum and Ethereum Classic, or between Bitcoin and Bitcoin Cash.
- lostcolony 5y ago
  You're also describing what Mnesia does in Erlang. But of course, you ARE playing around here - you're shifting the system(s) you're talking about when defining consistency. Consistency applies to the whole system, and just because you introduce a partition does not mean you now re-apply CAP considerations to the two systems independently, but still claim it applies to the original. You've changed your frame of reference. You either have one partitioned system that has given up consistency in favor of availability, or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial). Not some grammatical superposition of being both one and two systems.
  - derefr 5y ago
    > You're also describing what Mnesia does in Erlang. Not quite. Mnesia doesn't have any algorithm for automatic reconciliation; but it does expect reconciliation — in the sense that the idiomatic default thing to do is to have the individual netsplit partitions just halt and wait for a sysop to remsh in and manually reconcile them. (And this, because Mnesia is originally designed to be deployed in a leader/hot-standby setup, where only one node is ever accepting writes at a time, and there's automatic failover but no automatic failback if the original leader comes back up. The original leader just doesn't come back up, but rather comes up paused, waiting for a sysop to tell it what to do. Multi-master and/or automatic failback is out-of-design-scope for a DB designed for blades-on-a-shared-backplane network-switch control plane hardware.) Certainly, you can add code that tells the partitioned sub-clusters that "this is your life now", such that each cluster then boots up and continues on separately — but that's an explicit choice you're making, external to Mnesia's design. An empty callback is very different code than no callback. :) > or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial) Right, this is what I meant. And, given this viewpoint, the system is never "partitioned": it just becomes new systems. So you have sacrificed partition-tolerance — or rather, you've sacrificed partitionability. Your system can never enter a state where it has nodes that are expecting to ever reconnect to other nodes that might have missed updates, so your system can never be "partitioned" per se (just like a system with a single node can never be "partitioned" per se.)
    - lostcolony 5y ago
      Correct, it doesn't have automatic reconciliation, but you're phrasing things ambiguously that each individual partition just halts. If a node is starting up, and Mnesia detects that it had partitioned from the cluster previously, you are correct that it will halt until it is told what to do, because it reasonably assumes that it had entered a netsplit before going down, and the data can't be assumed to be consistent across all nodes. However, when a network partition first happens, even on reconnect, writes are still accepted on both sides of the split; they're operating in split brain fashion. And this is because during the partition it wasn't clear whether the other node went down, or was partitioned, ergo, it will accept writes to ensure the system stays up. It also is why the system will wait for direction on the restart case - if it didn't allow for writes during a split brain, it would never be inconsistent. In fact, if a node goes down without a split brain being first detected, and comes back up and rejoins the Erlang cluster, it will rejoin the Mnesia cluster, because it CAN determine where it is; it's at checkpoint 1, the cluster it's rejoining is at checkpoint 2, it can just accept all the writes that bring it up to checkpoint 2 without issue. And while, yes, the expectation generally is that you'll eventually heal the partition, the general expectation of that happening doesn't somehow change the CAP fundamentals. It's not even a universal expectation; depends what you're using Mnesia for; you could instead change the nodelist to enable each to function independently. The fact you have a use case for leaving them separate (as compared to the times it happens and you didn't, which still happens with a blockchain when a node shuts down or has network failure) doesn't change that. "You have sacrificed partitionability"; no, you haven't. You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system.
eloff 5y ago
Yeah, that wasn't the best choice of wording. Much prefer how you phrased it. It's really just about carefully defining the behavior of the system in the face of failures. Typically that means in a CP you tolerate some failures, but eventually you lose the majority and (perhaps only on one side of a network partition) and you have to sacrifice write availability at the minimum to maintain consistency - and reads might now be stale.

derefr 5y ago

> 'partition intolerant' partitionable system How about: any time there's a network partition, each partition decides on a new identity for itself, and becomes a new cluster that will never join back into the original, larger cluster again. People pick sides and never interact with people on the other side again. Consistency! (Because every node in each new smaller sub-cluster is now consistent with any other node said nodes will ever interact with again.) Availability! (Because all the nodes stay online.) Partition tolerance—in the sense of the partition ever being resolved? Nope. I'm not just playing around, either; I'm describing non-unilaterally-assented hard forks in blockchains — e.g. the split between Ethereum and Ethereum Classic, or between Bitcoin and Bitcoin Cash.

lostcolony 5y ago
You're also describing what Mnesia does in Erlang. But of course, you ARE playing around here - you're shifting the system(s) you're talking about when defining consistency. Consistency applies to the whole system, and just because you introduce a partition does not mean you now re-apply CAP considerations to the two systems independently, but still claim it applies to the original. You've changed your frame of reference. You either have one partitioned system that has given up consistency in favor of availability, or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial). Not some grammatical superposition of being both one and two systems.
- derefr 5y ago
  > You're also describing what Mnesia does in Erlang. Not quite. Mnesia doesn't have any algorithm for automatic reconciliation; but it does expect reconciliation — in the sense that the idiomatic default thing to do is to have the individual netsplit partitions just halt and wait for a sysop to remsh in and manually reconcile them. (And this, because Mnesia is originally designed to be deployed in a leader/hot-standby setup, where only one node is ever accepting writes at a time, and there's automatic failover but no automatic failback if the original leader comes back up. The original leader just doesn't come back up, but rather comes up paused, waiting for a sysop to tell it what to do. Multi-master and/or automatic failback is out-of-design-scope for a DB designed for blades-on-a-shared-backplane network-switch control plane hardware.) Certainly, you can add code that tells the partitioned sub-clusters that "this is your life now", such that each cluster then boots up and continues on separately — but that's an explicit choice you're making, external to Mnesia's design. An empty callback is very different code than no callback. :) > or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial) Right, this is what I meant. And, given this viewpoint, the system is never "partitioned": it just becomes new systems. So you have sacrificed partition-tolerance — or rather, you've sacrificed partitionability. Your system can never enter a state where it has nodes that are expecting to ever reconnect to other nodes that might have missed updates, so your system can never be "partitioned" per se (just like a system with a single node can never be "partitioned" per se.)
  - lostcolony 5y ago
    Correct, it doesn't have automatic reconciliation, but you're phrasing things ambiguously that each individual partition just halts. If a node is starting up, and Mnesia detects that it had partitioned from the cluster previously, you are correct that it will halt until it is told what to do, because it reasonably assumes that it had entered a netsplit before going down, and the data can't be assumed to be consistent across all nodes. However, when a network partition first happens, even on reconnect, writes are still accepted on both sides of the split; they're operating in split brain fashion. And this is because during the partition it wasn't clear whether the other node went down, or was partitioned, ergo, it will accept writes to ensure the system stays up. It also is why the system will wait for direction on the restart case - if it didn't allow for writes during a split brain, it would never be inconsistent. In fact, if a node goes down without a split brain being first detected, and comes back up and rejoins the Erlang cluster, it will rejoin the Mnesia cluster, because it CAN determine where it is; it's at checkpoint 1, the cluster it's rejoining is at checkpoint 2, it can just accept all the writes that bring it up to checkpoint 2 without issue. And while, yes, the expectation generally is that you'll eventually heal the partition, the general expectation of that happening doesn't somehow change the CAP fundamentals. It's not even a universal expectation; depends what you're using Mnesia for; you could instead change the nodelist to enable each to function independently. The fact you have a use case for leaving them separate (as compared to the times it happens and you didn't, which still happens with a blockchain when a node shuts down or has network failure) doesn't change that. "You have sacrificed partitionability"; no, you haven't. You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system.

eloff 5y ago

Yeah, that wasn't the best choice of wording. Much prefer how you phrased it. It's really just about carefully defining the behavior of the system in the face of failures. Typically that means in a CP you tolerate some failures, but eventually you lose the majority and (perhaps only on one side of a network partition) and you have to sacrifice write availability at the minimum to maintain consistency - and reads might now be stale.