4 ms·

You are mischaracterizing what TFA says. It says that partitions either keep a client from reaching any servers (in which case we don't care) or they keep some

You are mischaracterizing what TFA says. It says that partitions either keep a client from reaching any servers (in which case we don't care) or they keep some servers from talking to others (or some clients from talking to all servers), and that in the latter case whenever you can have a quorum (really, a majority of the servers) reachable then you can continue. You've completely elided the quorum thing to make it sound like TFA is nuts.

kstrauser 2y ago
Quorum doesn't let you ignore partitions. It can't. You still have to decide what happens to the nodes on the wrong side of the partition, or how to handle multiple partitions (you have 5 nodes. You have 2 netsplits. Now you have clusters of 2 nodes, 2 nodes, and 1 node. Quorum say what?). I don't think TFA is nuts. I do think the premise that engineers using cloud can ignore CAP theorem is wrong, though. It's a decision you must consider when you're designing a system. As long as everything is running well, it doesn't matter. You've got to make sure that the behavior when something breaks is what you want it to be. And you can't outsource that decision. It can't be worked around.
- cryptonector 2y ago
  > You still have to decide what happens to the nodes on the wrong side of the partition You don't because their clients won't also be isolated from the other servers. That's TFA's point, that in a cloud environment network partitions that affect clients generally deny them access to all servers (something you can't do anything about), and network partitions between servers only isolate some servers and none of the clients.
  - kstrauser 2y ago
    That’s fine if you value availability over consistency. Which is fine if that’s the appropriate choice for your use case! Even then you have to decide whether to accept updates in that situation, know that they may never be reconciled if the partition is permanent.
    - cryptonector 2y ago
      No, you can have availability and consistency as long as you have enough replicas. Network partitions that leave you w/o a quorum are like the whole cloud being down. Network partitions don't leave some clients only able to reach non-quorums -- this is the big point of TFA.
      - kstrauser 2y ago
        That presumes you can only have 1 partition at a time. That is not the case. You can optimize for good behavior when you end up with 2 disconnected graphs. It’s absolutely the case that a graph with N nodes can end up with N separate groups. Conceptually, imagine a single switch that connects all of them together and that switch dies. You can engineer cloud services to be resilient. AWS has done a great job of that in my personal experience. But worst cases can still happen. If you have an algorithm that runs in n*log(n) most of the time but 2^n sometimes, it can still be super useful. You have to prepare for the bad case though. If you say it’s been running great so far and we don’t worry about it, fine, but that doesn’t mean it can’t still blow up when it’s most inconvenient. It only means it hasn’t yet.
        cryptonector 2y ago
        Again, a partition that leaves you with no quorum anywhere is like the cloud being down.
        pessimizer 2y ago
        A partition that leaves you with no quorum anywhere is like the cloud being down if you value consistency over availability.
        cryptonector 2y ago
        Yes, but again, TFA says that you will trust the cloud will be up long enough that you would indeed prefer consistency. My goodness. So many commenters here today are completely ignoring TFA and giving lessons on CAP -- lessons that are correct, but which completely ignore that TFA is really arguing that clouds move the needle hard in one direction when considering the CAP trade-offs, to the point that "the CAP theorem is irrelevant for cloud systems" (which was this post's original title, though not TFA's).
  - bitwalker 2y ago
    Those aren't the only failure modes - you can have two sets of servers partitioned from one another (in two different data centers), both of which are reachable by clients. Do you allow those partitions to remain available, or do you stop accepting client connections until the partition heals? The "right" choice depends entirely on the application.
    - cryptonector 2y ago
      In TFA's world-view clients will not talk to isolated servers.
      - thayne 2y ago
        So that would be choosing consistency over availability.
        cryptonector 2y ago
        Yes. That's the point, that cloud changes the trade-off such that consistency becomes worthwhile.
  - ithkuil 2y ago
    Indeed and this is a very subtle point that I found very hard to explain when talking about availability in the event of network partitions. You have to consider the system as a whole including external clients since the definition of the availability of the entire system ultimately depends on the point of view of the end users of the system.
- jrumbut 2y ago
  It's poor timing for this article. The CrowdStrike incident has demonstrated that no matter what you do it is possible for any distributed system to be partitioned in such a way that you have to choose between consistency and availability and if the system is important then that choice is important.
  - darby_nine 2y ago
    That's true, but I think the catch is that most customers likely didn't view it as an investment that could decrease availability.
  - imtringued 2y ago
    You're ignoring that a system is temporarily partitioned for the duration of the time it takes to retrieve information from all nodes. Partition intolerance means you can only proceed when all nodes have been reached. This means partition intolerance not only deals with availability in the yes or no sense, but also in terms of latency.
- AtlasBarfed 2y ago
  Quorum just means you are a CP system rather than an AP system.
imtringued 2y ago
If you have a partition, then you have to wait until the partition resolves if you want consistency. Otherwise the other partition may receive an update that you are unaware of, that invalidates your consistency constraints. The author of the article doesn't actually understand the CAP theorem.

kstrauser 2y ago

Quorum doesn't let you ignore partitions. It can't. You still have to decide what happens to the nodes on the wrong side of the partition, or how to handle multiple partitions (you have 5 nodes. You have 2 netsplits. Now you have clusters of 2 nodes, 2 nodes, and 1 node. Quorum say what?). I don't think TFA is nuts. I do think the premise that engineers using cloud can ignore CAP theorem is wrong, though. It's a decision you must consider when you're designing a system. As long as everything is running well, it doesn't matter. You've got to make sure that the behavior when something breaks is what you want it to be. And you can't outsource that decision. It can't be worked around.

cryptonector 2y ago
> You still have to decide what happens to the nodes on the wrong side of the partition You don't because their clients won't also be isolated from the other servers. That's TFA's point, that in a cloud environment network partitions that affect clients generally deny them access to all servers (something you can't do anything about), and network partitions between servers only isolate some servers and none of the clients.
- kstrauser 2y ago
  That’s fine if you value availability over consistency. Which is fine if that’s the appropriate choice for your use case! Even then you have to decide whether to accept updates in that situation, know that they may never be reconciled if the partition is permanent.
  - cryptonector 2y ago
    No, you can have availability and consistency as long as you have enough replicas. Network partitions that leave you w/o a quorum are like the whole cloud being down. Network partitions don't leave some clients only able to reach non-quorums -- this is the big point of TFA.
    - kstrauser 2y ago
      That presumes you can only have 1 partition at a time. That is not the case. You can optimize for good behavior when you end up with 2 disconnected graphs. It’s absolutely the case that a graph with N nodes can end up with N separate groups. Conceptually, imagine a single switch that connects all of them together and that switch dies. You can engineer cloud services to be resilient. AWS has done a great job of that in my personal experience. But worst cases can still happen. If you have an algorithm that runs in n*log(n) most of the time but 2^n sometimes, it can still be super useful. You have to prepare for the bad case though. If you say it’s been running great so far and we don’t worry about it, fine, but that doesn’t mean it can’t still blow up when it’s most inconvenient. It only means it hasn’t yet.
      - cryptonector 2y ago
        Again, a partition that leaves you with no quorum anywhere is like the cloud being down.
        pessimizer 2y ago
        A partition that leaves you with no quorum anywhere is like the cloud being down if you value consistency over availability.
        cryptonector 2y ago
        Yes, but again, TFA says that you will trust the cloud will be up long enough that you would indeed prefer consistency. My goodness. So many commenters here today are completely ignoring TFA and giving lessons on CAP -- lessons that are correct, but which completely ignore that TFA is really arguing that clouds move the needle hard in one direction when considering the CAP trade-offs, to the point that "the CAP theorem is irrelevant for cloud systems" (which was this post's original title, though not TFA's).
- bitwalker 2y ago
  Those aren't the only failure modes - you can have two sets of servers partitioned from one another (in two different data centers), both of which are reachable by clients. Do you allow those partitions to remain available, or do you stop accepting client connections until the partition heals? The "right" choice depends entirely on the application.
  - cryptonector 2y ago
    In TFA's world-view clients will not talk to isolated servers.
    - thayne 2y ago
      So that would be choosing consistency over availability.
      - cryptonector 2y ago
        Yes. That's the point, that cloud changes the trade-off such that consistency becomes worthwhile.
- ithkuil 2y ago
  Indeed and this is a very subtle point that I found very hard to explain when talking about availability in the event of network partitions. You have to consider the system as a whole including external clients since the definition of the availability of the entire system ultimately depends on the point of view of the end users of the system.
jrumbut 2y ago
It's poor timing for this article. The CrowdStrike incident has demonstrated that no matter what you do it is possible for any distributed system to be partitioned in such a way that you have to choose between consistency and availability and if the system is important then that choice is important.
- darby_nine 2y ago
  That's true, but I think the catch is that most customers likely didn't view it as an investment that could decrease availability.
- imtringued 2y ago
  You're ignoring that a system is temporarily partitioned for the duration of the time it takes to retrieve information from all nodes. Partition intolerance means you can only proceed when all nodes have been reached. This means partition intolerance not only deals with availability in the yes or no sense, but also in terms of latency.
AtlasBarfed 2y ago
Quorum just means you are a CP system rather than an AP system.

imtringued 2y ago

If you have a partition, then you have to wait until the partition resolves if you want consistency. Otherwise the other partition may receive an update that you are unaware of, that invalidates your consistency constraints. The author of the article doesn't actually understand the CAP theorem.