5 ms·

Redundancy doesn't solve CAP. Even with a dozen connections you can end up with issues (A sent a packet, B received it, the connection you pick dropped the ack;

Redundancy doesn't solve CAP. Even with a dozen connections you can end up with issues (A sent a packet, B received it, the connection you pick dropped the ack; what happens?)

linuxhansl 5y ago
You try another connection (after some timeout)? Or you start N requests in parallel and then pick the first response?
- whateveracct 5y ago
  aren't retry policies inherently compromising on "A"? they fail and cause things to be slow when they take effect
- lostcolony 5y ago
  "After some timeout" - exactly. During that time you effectively have a partition; you have either accepted the write, meaning you are no longer consistent, or you are blocking on the write, meaning you are no longer available. If you have multiple, let's expand out to 3 instances (not necessary, but helps illustrate some of the issues better). A tells B something 10x. A also tells C something 10x. Even assuming you don't get inconsistent responses (i.e., B has both accepted and rejected the write), what happens if B accepts it, and C rejects it? Likewise, if you have high load, you're going to have race conditions leading to lack of consistency as well (A tells B and C x = 2, meanwhile C tells A and B that x = 3). You are favoring availability > consistency.
throwaway3699 5y ago
Then you have downtime. If the downtime is small enough, then the trade-off can be worthwhile.
- owenmarshall 5y ago
  You sacrificed A.
  - throwaway3699 5y ago
    So does Spanner. The point is that Spanner has so little downtime that it's not really an issue.
    - lostcolony 5y ago
      Spanner is a strongly consistent system leveraging Paxos. Yes, with sufficient redundancy you can reduce the likelihood of a partition. You still want to know what the guarantees of the system are. Spanner guarantees that if a transaction can't be agreed to by a quorum of nodes, it will be rejected. Ergo, favoring consistency. That guarantee matters, no matter how infrequent partitions are.
- lostcolony 5y ago
  So...not available.
toast0 5y ago
> Redundancy doesn't solve CAP. Even with a dozen connections you can end up with issues (A sent a packet, B received it, the connection you pick dropped the ack; what happens?) If you really need things to arrive, you'd be sending them on more than one connection. Of course, the difficulty with redundant fiber connections is that really the only way to determine if redundant fiber connections are in the same bundle is with a backhoe.

Redundancy doesn't solve CAP. Even with a dozen connections you can end up with issues (A sent a packet, B received it, the connection you pick dropped the ack; what happens?)

linuxhansl 5y ago

You try another connection (after some timeout)? Or you start N requests in parallel and then pick the first response?

whateveracct 5y ago
aren't retry policies inherently compromising on "A"? they fail and cause things to be slow when they take effect
lostcolony 5y ago
"After some timeout" - exactly. During that time you effectively have a partition; you have either accepted the write, meaning you are no longer consistent, or you are blocking on the write, meaning you are no longer available. If you have multiple, let's expand out to 3 instances (not necessary, but helps illustrate some of the issues better). A tells B something 10x. A also tells C something 10x. Even assuming you don't get inconsistent responses (i.e., B has both accepted and rejected the write), what happens if B accepts it, and C rejects it? Likewise, if you have high load, you're going to have race conditions leading to lack of consistency as well (A tells B and C x = 2, meanwhile C tells A and B that x = 3). You are favoring availability > consistency.

throwaway3699 5y ago

Then you have downtime. If the downtime is small enough, then the trade-off can be worthwhile.

owenmarshall 5y ago
You sacrificed A.
- throwaway3699 5y ago
  So does Spanner. The point is that Spanner has so little downtime that it's not really an issue.
  - lostcolony 5y ago
    Spanner is a strongly consistent system leveraging Paxos. Yes, with sufficient redundancy you can reduce the likelihood of a partition. You still want to know what the guarantees of the system are. Spanner guarantees that if a transaction can't be agreed to by a quorum of nodes, it will be rejected. Ergo, favoring consistency. That guarantee matters, no matter how infrequent partitions are.
lostcolony 5y ago
So...not available.

toast0 5y ago

> Redundancy doesn't solve CAP. Even with a dozen connections you can end up with issues (A sent a packet, B received it, the connection you pick dropped the ack; what happens?) If you really need things to arrive, you'd be sending them on more than one connection. Of course, the difficulty with redundant fiber connections is that really the only way to determine if redundant fiber connections are in the same bundle is with a backhoe.