4 ms·

I thought we had worked out in 2010-2012 that the CAP theorem, consistency, availability and partition tolerance (https://en.wikipedia.org/wiki/CAP_theorem#Hist

I thought we had worked out in 2010-2012 that the CAP theorem, consistency, availability and partition tolerance (https://en.wikipedia.org/wiki/CAP_theorem#History https://en.wikipedia.org/wiki/CAP_theorem#History) made this impossible? Where is the mistake.

chillydawg 6y ago
I would guess write times are slightly up to cover the cost of the new replication completion guarantees.
johncolanduoni 6y ago
S3 is almost certainly not fully partition tolerant at the node level and requires some sort of quorum. Other “magical” data stores like Spanner also retain this limitation, they just have very reliable replication strategies.
- pricechild 6y ago
  "almost certainly not fully partition tolerant at the node level" Whoa hang on! You can't just say you're not tolerant of partitions... I'm struggling to find the best post on aphyr.com about this but https://aphyr.com/posts/325-comments-on-you-do-it-to https://aphyr.com/posts/325-comments-on-you-do-it-to is a good one, specifically the line: "CP and AP are upper bounds: systems can provide C or A during a partition, but might provide neither." In short, Partitions happen, no matter what. No really, when you have multiple nodes they happen, you must tolerate them. You can't have CA.
  - johncolanduoni 6y ago
    I’m not claiming it’s a CA system, and the terminology “partition intolerant” is not verboten by Kyle Kingsbury. From your link: “Specifically, partition-intolerant systems must sacrifice invariants when partitions occur. Which invariants?” The answer in this case is that availability is sacrificed, unless Amazon is making a very misleading claim of strong consistency (per the submission title/link). So it’s CP. In case “not fully partition tolerant” is what is throwing you off, all I mean by that is that there are likely network partitions of the S3 nodes that will not cause outward failure (e.g. partition of a single node), though some will (e.g. too many partitions to form a quorum, or whatever their underlying consensus implementation relies on).
    - throw93 6y ago
      >> Amazon S3 delivers strong read-after-write consistency automatically for all applications, without changes to performance or availability, without sacrificing regional isolation for applications, and at no additional cost. So they claim performance and availability will remain same while claiming strong consistency. I was confused at first but then “same” availability isn’t 100% availability. So it indeed CP.
    - pricechild 6y ago
      Cool, sorry to put words in your mouth. I guess the simple answer to the GP is "Availability is likely sacrificed", and the clue's in the submission title being all about consistency. I haven't looked too deep into it and am sure there are also still Consistency sacrifices too. I'm still a little niggled though... "almost certainly not fully partition tolerant at the node level and requires some sort of quorum" - what does this actually mean? I'm confused why you raise the tolerance of a single node to partitions, when the partitions by definition require >1 node?
      - johncolanduoni 6y ago
        When I say “at the node level” I’m trying to draw a distinction with the service level view where a client is talking to S3 as a (from their perspective) single entity per region. So a better way of putting it may be the internal view of the service, where partitions are between nodes that are part of S3 itself (as opposed to between a client and S3) and they affect the overall health of the system. I’m not referring to a single node being partition resistant (hence why I said node level, and never mentioned single) which as you said wouldn’t make any sense.

chillydawg 6y ago

I would guess write times are slightly up to cover the cost of the new replication completion guarantees.

johncolanduoni 6y ago

S3 is almost certainly not fully partition tolerant at the node level and requires some sort of quorum. Other “magical” data stores like Spanner also retain this limitation, they just have very reliable replication strategies.

pricechild 6y ago
"almost certainly not fully partition tolerant at the node level" Whoa hang on! You can't just say you're not tolerant of partitions... I'm struggling to find the best post on aphyr.com about this but https://aphyr.com/posts/325-comments-on-you-do-it-to https://aphyr.com/posts/325-comments-on-you-do-it-to is a good one, specifically the line: "CP and AP are upper bounds: systems can provide C or A during a partition, but might provide neither." In short, Partitions happen, no matter what. No really, when you have multiple nodes they happen, you must tolerate them. You can't have CA.
- johncolanduoni 6y ago
  I’m not claiming it’s a CA system, and the terminology “partition intolerant” is not verboten by Kyle Kingsbury. From your link: “Specifically, partition-intolerant systems must sacrifice invariants when partitions occur. Which invariants?” The answer in this case is that availability is sacrificed, unless Amazon is making a very misleading claim of strong consistency (per the submission title/link). So it’s CP. In case “not fully partition tolerant” is what is throwing you off, all I mean by that is that there are likely network partitions of the S3 nodes that will not cause outward failure (e.g. partition of a single node), though some will (e.g. too many partitions to form a quorum, or whatever their underlying consensus implementation relies on).
  - throw93 6y ago
    >> Amazon S3 delivers strong read-after-write consistency automatically for all applications, without changes to performance or availability, without sacrificing regional isolation for applications, and at no additional cost. So they claim performance and availability will remain same while claiming strong consistency. I was confused at first but then “same” availability isn’t 100% availability. So it indeed CP.
  - pricechild 6y ago
    Cool, sorry to put words in your mouth. I guess the simple answer to the GP is "Availability is likely sacrificed", and the clue's in the submission title being all about consistency. I haven't looked too deep into it and am sure there are also still Consistency sacrifices too. I'm still a little niggled though... "almost certainly not fully partition tolerant at the node level and requires some sort of quorum" - what does this actually mean? I'm confused why you raise the tolerance of a single node to partitions, when the partitions by definition require >1 node?
    - johncolanduoni 6y ago
      When I say “at the node level” I’m trying to draw a distinction with the service level view where a client is talking to S3 as a (from their perspective) single entity per region. So a better way of putting it may be the internal view of the service, where partitions are between nodes that are part of S3 itself (as opposed to between a client and S3) and they affect the overall health of the system. I’m not referring to a single node being partition resistant (hence why I said node level, and never mentioned single) which as you said wouldn’t make any sense.