6 ms·

I think the author, like many recently exposed to the CAP theorem, is confused about the meaning of partition tolerance, leading to ridiculous conclusions. Par

by dadkins 16y ago

I think the author, like many recently exposed to the CAP theorem, is confused about the meaning of partition tolerance, leading to ridiculous conclusions. Partition tolerance does not mean your distributed system can't be consistent and available because your network dropped one packet, or one node failed. What would be the point of such a definition? Instead, the CAP theorem implies that while the network is partitioned, consistency or availability must be sacrificed. In the case of the dropped packet, once it is retransmitted the partition is healed and progress can be made. Or in the case of the failed node, nothing says that the rest of the system can't be consistent and available, so that the system as a whole maintains that property. There is no requirement that the unavailable node be available. Truly partition tolerant systems are those that continue to function in the face of a prolonged partition, and those are the systems that must sacrifice either consistency or availability.

allertonm 16y ago
What you're saying is that so long as there are no partitions, the system can be Consistent and Available, but if there's a partition, it can't. "Consistent sometimes" is not the same thing as "Consistent" and "Available sometimes" is not the same thing as "Available" - and so "Consistent and Available sometimes" is not the same as "Consistent and Available". I believe you might be guilty of confusing "Eventual Consistency" with "Consistency". Funnily enough, no-one has found much use for "Eventual Availability" so far.
- mcodik 16y ago
  Assuming eventual availability can be pretty handy -- one way deal with a dependency outage is to retry with an exponential backoff. If a dependency is unavailable now, and your system keeps retrying until it is, then you are assuming your dependency will become available again eventually.
  - allertonm 16y ago
    Fair point - but of course your client is not making any progress, and so the unavailability ripples up. It's unlikely that there is user facing case where this is a useful way to work, though I can see it's use in loosely coupled connections between backends.
- dadkins 16y ago
  Not quite. What I'm saying is that a dropped packet or a failed node are not partitions as far as the CAP theorem is considered. A distributed system is considered available if "every request received by a non-failing node [results] in a response." It does not mean you cannot retransmit or retry. Similarly, the consistency guarantee only requires that there exist a total order on operations. Failures are ok, as we're allowed to retransmit, retry, and otherwise tolerate faults. There is no inconsistency, nor is anything eventual. My point is that a "temporary" partition is just a fault, and as long as the fault is shorter than the allowed response time of the system, it doesn't make a difference.
  - allertonm 16y ago
    Yes, a system is available if one node doesn't respond and you can contact another. But that node will be unable to guarantee consistency. If a node you can contact is required to guarantee consistency, there will be some times that it will have to refuse your request because other nodes are not contactable. The author's point was that in any distributed system there is a non-zero probability of a network failure. While both clients and server nodes can retry connections, there is a non-zero probability that the problem will persist longer than your "availability agreement" allows. In that case, you have a choice - return potentially inconstent data or refuse the request. What you seem to be arguing is that the probabilities of failure - in particular of repeated failure - while non-zero, are effectively zero. The author would disagree (as they point out, the probabilities combine exponentially as the number of nodes increase.) I think he's right and that you are wrong.
    - dadkins 16y ago
      Paxos is the quintessential example of a highly available, consistent system. It is available as long as more than half of the nodes are up and able to communicate with each other. It remains consistent, regardless of the failure pattern. You really do only have to worry about a true network partition. This isn't a probabilistic argument in any sense.
      - allertonm 16y ago
HenryR 16y ago
I'm afraid you're not quite correct. CAP says that, in the face of potentially arbitrary network partitions (which are precisely modelled by a set of dropped messages) you can be 100% consistent, or 100% available, but you can't be both. If the network is allowed to drop packets there are times where either you must not respond to requests (as doing so would violate sequential consistency) or you must respond incorrectly, potentially with stale information. The network partition that forces you to drop one of these guarantees might be quite dramatic - but note that, for example, a quorum system is only available on the majority side of its partition. Therefore if a single node is unable to deliver messages (due to a network partition event), it will not be able to correctly respond to requests and must either not respond to its clients or respond incorrectly.
- tbrownaw 16y ago
  > I'm afraid you're not quite correct. So then what am I missing? It sounds to me like the rest of your post is reiterating what you just called incorrect.
  - technomancy 16y ago
    You said that wasn't what TFA was describing, but it is.

by dadkins 16y ago

allertonm 16y ago

What you're saying is that so long as there are no partitions, the system can be Consistent and Available, but if there's a partition, it can't. "Consistent sometimes" is not the same thing as "Consistent" and "Available sometimes" is not the same thing as "Available" - and so "Consistent and Available sometimes" is not the same as "Consistent and Available". I believe you might be guilty of confusing "Eventual Consistency" with "Consistency". Funnily enough, no-one has found much use for "Eventual Availability" so far.

mcodik 16y ago
Assuming eventual availability can be pretty handy -- one way deal with a dependency outage is to retry with an exponential backoff. If a dependency is unavailable now, and your system keeps retrying until it is, then you are assuming your dependency will become available again eventually.
- allertonm 16y ago
  Fair point - but of course your client is not making any progress, and so the unavailability ripples up. It's unlikely that there is user facing case where this is a useful way to work, though I can see it's use in loosely coupled connections between backends.
dadkins 16y ago
Not quite. What I'm saying is that a dropped packet or a failed node are not partitions as far as the CAP theorem is considered. A distributed system is considered available if "every request received by a non-failing node [results] in a response." It does not mean you cannot retransmit or retry. Similarly, the consistency guarantee only requires that there exist a total order on operations. Failures are ok, as we're allowed to retransmit, retry, and otherwise tolerate faults. There is no inconsistency, nor is anything eventual. My point is that a "temporary" partition is just a fault, and as long as the fault is shorter than the allowed response time of the system, it doesn't make a difference.
- allertonm 16y ago
  Yes, a system is available if one node doesn't respond and you can contact another. But that node will be unable to guarantee consistency. If a node you can contact is required to guarantee consistency, there will be some times that it will have to refuse your request because other nodes are not contactable. The author's point was that in any distributed system there is a non-zero probability of a network failure. While both clients and server nodes can retry connections, there is a non-zero probability that the problem will persist longer than your "availability agreement" allows. In that case, you have a choice - return potentially inconstent data or refuse the request. What you seem to be arguing is that the probabilities of failure - in particular of repeated failure - while non-zero, are effectively zero. The author would disagree (as they point out, the probabilities combine exponentially as the number of nodes increase.) I think he's right and that you are wrong.
  - dadkins 16y ago
    Paxos is the quintessential example of a highly available, consistent system. It is available as long as more than half of the nodes are up and able to communicate with each other. It remains consistent, regardless of the failure pattern. You really do only have to worry about a true network partition. This isn't a probabilistic argument in any sense.
    - allertonm 16y ago

HenryR 16y ago

I'm afraid you're not quite correct. CAP says that, in the face of potentially arbitrary network partitions (which are precisely modelled by a set of dropped messages) you can be 100% consistent, or 100% available, but you can't be both. If the network is allowed to drop packets there are times where either you must not respond to requests (as doing so would violate sequential consistency) or you must respond incorrectly, potentially with stale information. The network partition that forces you to drop one of these guarantees might be quite dramatic - but note that, for example, a quorum system is only available on the majority side of its partition. Therefore if a single node is unable to deliver messages (due to a network partition event), it will not be able to correctly respond to requests and must either not respond to its clients or respond incorrectly.

tbrownaw 16y ago
> I'm afraid you're not quite correct. So then what am I missing? It sounds to me like the rest of your post is reiterating what you just called incorrect.
- technomancy 16y ago
  You said that wasn't what TFA was describing, but it is.