3 ms·

You're also describing what Mnesia does in Erlang. But of course, you ARE playing around here - you're shifting the system(s) you're talking about when definin

You're also describing what Mnesia does in Erlang. But of course, you ARE playing around here - you're shifting the system(s) you're talking about when defining consistency. Consistency applies to the whole system, and just because you introduce a partition does not mean you now re-apply CAP considerations to the two systems independently, but still claim it applies to the original. You've changed your frame of reference. You either have one partitioned system that has given up consistency in favor of availability, or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial). Not some grammatical superposition of being both one and two systems.

derefr 5y ago
> You're also describing what Mnesia does in Erlang. Not quite. Mnesia doesn't have any algorithm for automatic reconciliation; but it does expect reconciliation — in the sense that the idiomatic default thing to do is to have the individual netsplit partitions just halt and wait for a sysop to remsh in and manually reconcile them. (And this, because Mnesia is originally designed to be deployed in a leader/hot-standby setup, where only one node is ever accepting writes at a time, and there's automatic failover but no automatic failback if the original leader comes back up. The original leader just doesn't come back up, but rather comes up paused, waiting for a sysop to tell it what to do. Multi-master and/or automatic failback is out-of-design-scope for a DB designed for blades-on-a-shared-backplane network-switch control plane hardware.) Certainly, you can add code that tells the partitioned sub-clusters that "this is your life now", such that each cluster then boots up and continues on separately — but that's an explicit choice you're making, external to Mnesia's design. An empty callback is very different code than no callback. :) > or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial) Right, this is what I meant. And, given this viewpoint, the system is never "partitioned": it just becomes new systems. So you have sacrificed partition-tolerance — or rather, you've sacrificed partitionability. Your system can never enter a state where it has nodes that are expecting to ever reconnect to other nodes that might have missed updates, so your system can never be "partitioned" per se (just like a system with a single node can never be "partitioned" per se.)
- lostcolony 5y ago
  Correct, it doesn't have automatic reconciliation, but you're phrasing things ambiguously that each individual partition just halts. If a node is starting up, and Mnesia detects that it had partitioned from the cluster previously, you are correct that it will halt until it is told what to do, because it reasonably assumes that it had entered a netsplit before going down, and the data can't be assumed to be consistent across all nodes. However, when a network partition first happens, even on reconnect, writes are still accepted on both sides of the split; they're operating in split brain fashion. And this is because during the partition it wasn't clear whether the other node went down, or was partitioned, ergo, it will accept writes to ensure the system stays up. It also is why the system will wait for direction on the restart case - if it didn't allow for writes during a split brain, it would never be inconsistent. In fact, if a node goes down without a split brain being first detected, and comes back up and rejoins the Erlang cluster, it will rejoin the Mnesia cluster, because it CAN determine where it is; it's at checkpoint 1, the cluster it's rejoining is at checkpoint 2, it can just accept all the writes that bring it up to checkpoint 2 without issue. And while, yes, the expectation generally is that you'll eventually heal the partition, the general expectation of that happening doesn't somehow change the CAP fundamentals. It's not even a universal expectation; depends what you're using Mnesia for; you could instead change the nodelist to enable each to function independently. The fact you have a use case for leaving them separate (as compared to the times it happens and you didn't, which still happens with a blockchain when a node shuts down or has network failure) doesn't change that. "You have sacrificed partitionability"; no, you haven't. You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system.
  - derefr 5y ago
    > You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system. I'm very confused. If there's a new system, and the old system doesn't exist any more, then how is the old system partitioned? The old system is down. The new system just happens to contain nodes from the old system. (This is a crucial distinction: if you tried to introduce your new node to the sub-cluster on either side of the netsplit in this model, while advertising yourself as being a participant in the parent network they broke off from, all new sub-networks would refuse to connect to you. None of the sub-networks consider themselves a continuation of the parent. Each netsplit gives you two new networks.) Saying this kind of permanent intended split is "partitioning", is like saying deallocated memory gets "corrupted" when new memory allocates over it. It only makes sense to say such a thing in a world where you didn't do the thing you did (deallocate the memory), because by doing so, you removed the semantics under which corruption makes sense to talk about. The allocated struct visible to your process is gone, and so there's no longer a requirement from your program that it be in any particular state. So, whatever state it's in, that's "just fine" as far as your program is concerned. Full of 0xdeadbeef? Valid! Because there's no longer any meaning to validity, past deallocation. A partitioned system is a system that cares about being partitioned — and whose operators care about it being partitioned. Being partitioned is a state that exists in the context of the design goals of the architecture of the system, and an implementation as a control system that attempts to achieve those goals. Two random nodes that have never talked to one-another aren't "partitioned" — they're just not part of a shared network-architecture design that would make partitioning between them a valid thing to talk about. They've never tried to communicate; they have no reified logic that says they should be communicating. By designing a distributed system that can never care about nodes that leave it — that can never model other nodes as ever being able to rejoin, such that they could ever potentially introduce conflicting updates — you thereby design a system that exists outside the semantics for which being "partitioned" is well-defined as a concept. You're sacrificing "the ability to have partitions as part of your domain model", in the same sense that deallocation sacrifices "the ability to have structural validity as part of your domain model."

derefr 5y ago

> You're also describing what Mnesia does in Erlang. Not quite. Mnesia doesn't have any algorithm for automatic reconciliation; but it does expect reconciliation — in the sense that the idiomatic default thing to do is to have the individual netsplit partitions just halt and wait for a sysop to remsh in and manually reconcile them. (And this, because Mnesia is originally designed to be deployed in a leader/hot-standby setup, where only one node is ever accepting writes at a time, and there's automatic failover but no automatic failback if the original leader comes back up. The original leader just doesn't come back up, but rather comes up paused, waiting for a sysop to tell it what to do. Multi-master and/or automatic failback is out-of-design-scope for a DB designed for blades-on-a-shared-backplane network-switch control plane hardware.) Certainly, you can add code that tells the partitioned sub-clusters that "this is your life now", such that each cluster then boots up and continues on separately — but that's an explicit choice you're making, external to Mnesia's design. An empty callback is very different code than no callback. :) > or you have two systems, neither of which are partitioned (and which from the purposes of CAP are unrelated, since the initial source of seed data for the system is immaterial) Right, this is what I meant. And, given this viewpoint, the system is never "partitioned": it just becomes new systems. So you have sacrificed partition-tolerance — or rather, you've sacrificed partitionability. Your system can never enter a state where it has nodes that are expecting to ever reconnect to other nodes that might have missed updates, so your system can never be "partitioned" per se (just like a system with a single node can never be "partitioned" per se.)

lostcolony 5y ago
Correct, it doesn't have automatic reconciliation, but you're phrasing things ambiguously that each individual partition just halts. If a node is starting up, and Mnesia detects that it had partitioned from the cluster previously, you are correct that it will halt until it is told what to do, because it reasonably assumes that it had entered a netsplit before going down, and the data can't be assumed to be consistent across all nodes. However, when a network partition first happens, even on reconnect, writes are still accepted on both sides of the split; they're operating in split brain fashion. And this is because during the partition it wasn't clear whether the other node went down, or was partitioned, ergo, it will accept writes to ensure the system stays up. It also is why the system will wait for direction on the restart case - if it didn't allow for writes during a split brain, it would never be inconsistent. In fact, if a node goes down without a split brain being first detected, and comes back up and rejoins the Erlang cluster, it will rejoin the Mnesia cluster, because it CAN determine where it is; it's at checkpoint 1, the cluster it's rejoining is at checkpoint 2, it can just accept all the writes that bring it up to checkpoint 2 without issue. And while, yes, the expectation generally is that you'll eventually heal the partition, the general expectation of that happening doesn't somehow change the CAP fundamentals. It's not even a universal expectation; depends what you're using Mnesia for; you could instead change the nodelist to enable each to function independently. The fact you have a use case for leaving them separate (as compared to the times it happens and you didn't, which still happens with a blockchain when a node shuts down or has network failure) doesn't change that. "You have sacrificed partitionability"; no, you haven't. You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system.
- derefr 5y ago
  > You've intentionally partitioned the original system, allowing them to be inconsistent, and choosing one side of the partition to promote to being a new system. I'm very confused. If there's a new system, and the old system doesn't exist any more, then how is the old system partitioned? The old system is down. The new system just happens to contain nodes from the old system. (This is a crucial distinction: if you tried to introduce your new node to the sub-cluster on either side of the netsplit in this model, while advertising yourself as being a participant in the parent network they broke off from, all new sub-networks would refuse to connect to you. None of the sub-networks consider themselves a continuation of the parent. Each netsplit gives you two new networks.) Saying this kind of permanent intended split is "partitioning", is like saying deallocated memory gets "corrupted" when new memory allocates over it. It only makes sense to say such a thing in a world where you didn't do the thing you did (deallocate the memory), because by doing so, you removed the semantics under which corruption makes sense to talk about. The allocated struct visible to your process is gone, and so there's no longer a requirement from your program that it be in any particular state. So, whatever state it's in, that's "just fine" as far as your program is concerned. Full of 0xdeadbeef? Valid! Because there's no longer any meaning to validity, past deallocation. A partitioned system is a system that cares about being partitioned — and whose operators care about it being partitioned. Being partitioned is a state that exists in the context of the design goals of the architecture of the system, and an implementation as a control system that attempts to achieve those goals. Two random nodes that have never talked to one-another aren't "partitioned" — they're just not part of a shared network-architecture design that would make partitioning between them a valid thing to talk about. They've never tried to communicate; they have no reified logic that says they should be communicating. By designing a distributed system that can never care about nodes that leave it — that can never model other nodes as ever being able to rejoin, such that they could ever potentially introduce conflicting updates — you thereby design a system that exists outside the semantics for which being "partitioned" is well-defined as a concept. You're sacrificing "the ability to have partitions as part of your domain model", in the same sense that deallocation sacrifices "the ability to have structural validity as part of your domain model."