5 ms·

It's interesting to read all these comments here that talk about the eventual consistency like it was some kind of bug.

by k__ 6y ago

howlgarnish 6y ago
Are there any scenarios where you would want eventual consistency over strong consistency? (Assuming pricing, performance, replication etc are the same.)
- crazyjncsu 6y ago
  No.
- nhumrich 6y ago
  At the benefit of added availability? absolutely. This is the entire premise of mongodb.
  - nullsense 6y ago
    MongoDB is webscale.
    - castlecrasher2 6y ago
      I want the one with the bigger GBs.
      - nullsense 6y ago
        Better yet, just pipe all your data to /dev/null it supports sharding. Sharding is the secret in the webscale sauce.
  - oblio 6y ago
    Availability? I thought Mongo was fast and scalable, I don't remember anyone recommending it for higher availability.
    - nhumrich 6y ago
      > Fast and scalable > Availability These are synonyms. How do you scale? Horizontally. How you do it without degrading performance, is the same way you increase availability. Eventual consistency.
- TheCoelacanth 6y ago
  Job security?
- dboreham 6y ago
  Train tunnels
- acdha 6y ago
  Your last sentence is the answer I’ve always heard given: this in the context of a performance trade off — do you wait for every node in the system to acknowledge an update and purge caches, or return as soon as a sufficient durability threshold has been hit?
- robotresearcher 6y ago
  No. A magical do-everything datastore would have strong consistency. Eventual consistency is something you might settle for given the performance cost trade offs. To see this, consider that strong consistency has all the guarantees of eventual consistency, plus some more. And the additional guarantees might make your application much easier to write.
deathanatos 6y ago
The previous behavior (new keys weren't strongly consistent) were weird, and made a lot of patterns that AWS itself sort of guides you towards, broken. For example, you can set up an SQS queue to be informed of writes into a bucket. The idea being that some component writes into the bucket, and a consumer then fetches the data & processes it. Except — gotcha! — there wasn't a guarantee of consistency. It would usually work, though, and so you'd get these weird failures that you couldn't explain unless you already knew the answer, really. Same thing with how S3 object names sort of feel like file names, and that you might use them for something useful, and you design around that, and then you hit the section of the docs that says "no, you shouldn't, they should be uniformly balanced". (Though honestly, I've never felt any negative side-effects from ignoring this, unlike the consistency guarantees.)
- jsjohnst 6y ago
  > then you hit the section of the docs that says "no, you shouldn't, they should be uniformly balanced". (Though honestly, I've never felt any negative side-effects from ignoring this You likely won’t, unless you hit a bucket size that has at least a billion small files inside. If you get into the trillion and above range and are heavily unbalanced due to a recent deploy of yours, ooops, there goes S3. Or it did in 2013 anyway.
- 0xbadcafebee 6y ago
  So the problem was that users don't know how S3 works?
CobrastanJorji 6y ago
If someone uses your API wrong, that's their bug. If a huge number of people use your API wrong, that's your bug.
zwily 6y ago
It wasn’t a bug, it was well-documented behavior. However, it was the root of many bugs by people building on top of S3 that did not take the eventual consistency into account. I’ll bet this release is going to fix a lot of weird bugs in systems out there using S3.
- k__ 6y ago
  Guess the name simple storage service gave many people the idea that you could use it without a basic idea of distributed systems.
SilasX 6y ago
I'm pulling my hair out because people are using eventual consistency to mean "eventual consistency with a noticeable lag before the consistency is achieved", which bothers the pedant in me. I mean, technically, strong consistency implies eventual consistency (with lag = 0). But everyone's equating eventual consistency with the noticeable lag itself, implying that EC per se is a bad thing. For an analogy, it would be like if people were talking about how rectangular cakes suck, because <problems of unequal width vs length>, and thus they use square cakes, but ... square cakes are rectangular too. (What's going on is that people use <larger set> to mean <larger set, excluding members that qualify for smaller set>. <larger set> = eventual consistency, rectangle; <smaller set> = strong consistency, square.) Obviously, it's not a problem for communication because everyone implicitly understands what's going on and uses the words the same way, but I wish people spoke in terms of "has anyone actually seen a consistency lag?" rather than "has anyone actually seen eventual consistency?", since the latter is not the right way to frame it, and is actually a good thing to have, which happened both before and after this development.
- benlivengood 6y ago
  Eventual consistency with zero lag is not strong consistency. Two simultaneous conflicting writes with eventual consistency will appear successful. Strong consistency guarantees that one of the writes will fail. EDIT: I think the reason people care about the lag is specifically for this case; if you want to know if your eventually consistent write actually succeeded then you need to know how long to wait to check.
  - SilasX 6y ago
    Well, in all the threads, they’re describing situations with lag. That is, they’re equating eventual consistently with the lag, and implying it’s therefore bad, when eventual consistency can include unnoticeably small lag. And thus it’s the lag that they’re actually bothered by, not the EC per se. Hence my clawed out hair.

It's interesting to read all these comments here that talk about the eventual consistency like it was some kind of bug.

by k__ 6y ago

It's interesting to read all these comments here that talk about the eventual consistency like it was some kind of bug.

Are there any scenarios where you would want eventual consistency over strong consistency? (Assuming pricing, performance, replication etc are the same.)

crazyjncsu 6y ago
No.
nhumrich 6y ago
At the benefit of added availability? absolutely. This is the entire premise of mongodb.
- nullsense 6y ago
  MongoDB is webscale.
  - castlecrasher2 6y ago
    I want the one with the bigger GBs.
    - nullsense 6y ago
      Better yet, just pipe all your data to /dev/null it supports sharding. Sharding is the secret in the webscale sauce.
- oblio 6y ago
  Availability? I thought Mongo was fast and scalable, I don't remember anyone recommending it for higher availability.
  - nhumrich 6y ago
    > Fast and scalable > Availability These are synonyms. How do you scale? Horizontally. How you do it without degrading performance, is the same way you increase availability. Eventual consistency.
TheCoelacanth 6y ago
Job security?
dboreham 6y ago
Train tunnels
acdha 6y ago
Your last sentence is the answer I’ve always heard given: this in the context of a performance trade off — do you wait for every node in the system to acknowledge an update and purge caches, or return as soon as a sufficient durability threshold has been hit?
robotresearcher 6y ago
No. A magical do-everything datastore would have strong consistency. Eventual consistency is something you might settle for given the performance cost trade offs. To see this, consider that strong consistency has all the guarantees of eventual consistency, plus some more. And the additional guarantees might make your application much easier to write.

deathanatos 6y ago

The previous behavior (new keys weren't strongly consistent) were weird, and made a lot of patterns that AWS itself sort of guides you towards, broken. For example, you can set up an SQS queue to be informed of writes into a bucket. The idea being that some component writes into the bucket, and a consumer then fetches the data & processes it. Except — gotcha! — there wasn't a guarantee of consistency. It would usually work, though, and so you'd get these weird failures that you couldn't explain unless you already knew the answer, really. Same thing with how S3 object names sort of feel like file names, and that you might use them for something useful, and you design around that, and then you hit the section of the docs that says "no, you shouldn't, they should be uniformly balanced". (Though honestly, I've never felt any negative side-effects from ignoring this, unlike the consistency guarantees.)

jsjohnst 6y ago
> then you hit the section of the docs that says "no, you shouldn't, they should be uniformly balanced". (Though honestly, I've never felt any negative side-effects from ignoring this You likely won’t, unless you hit a bucket size that has at least a billion small files inside. If you get into the trillion and above range and are heavily unbalanced due to a recent deploy of yours, ooops, there goes S3. Or it did in 2013 anyway.
0xbadcafebee 6y ago
So the problem was that users don't know how S3 works?

CobrastanJorji 6y ago

If someone uses your API wrong, that's their bug. If a huge number of people use your API wrong, that's your bug.

zwily 6y ago

It wasn’t a bug, it was well-documented behavior. However, it was the root of many bugs by people building on top of S3 that did not take the eventual consistency into account. I’ll bet this release is going to fix a lot of weird bugs in systems out there using S3.

k__ 6y ago
Guess the name simple storage service gave many people the idea that you could use it without a basic idea of distributed systems.

SilasX 6y ago

I'm pulling my hair out because people are using eventual consistency to mean "eventual consistency with a noticeable lag before the consistency is achieved", which bothers the pedant in me. I mean, technically, strong consistency implies eventual consistency (with lag = 0). But everyone's equating eventual consistency with the noticeable lag itself, implying that EC per se is a bad thing. For an analogy, it would be like if people were talking about how rectangular cakes suck, because <problems of unequal width vs length>, and thus they use square cakes, but ... square cakes are rectangular too. (What's going on is that people use <larger set> to mean <larger set, excluding members that qualify for smaller set>. <larger set> = eventual consistency, rectangle; <smaller set> = strong consistency, square.) Obviously, it's not a problem for communication because everyone implicitly understands what's going on and uses the words the same way, but I wish people spoke in terms of "has anyone actually seen a consistency lag?" rather than "has anyone actually seen eventual consistency?", since the latter is not the right way to frame it, and is actually a good thing to have, which happened both before and after this development.

benlivengood 6y ago
Eventual consistency with zero lag is not strong consistency. Two simultaneous conflicting writes with eventual consistency will appear successful. Strong consistency guarantees that one of the writes will fail. EDIT: I think the reason people care about the lag is specifically for this case; if you want to know if your eventually consistent write actually succeeded then you need to know how long to wait to check.
- SilasX 6y ago
  Well, in all the threads, they’re describing situations with lag. That is, they’re equating eventual consistently with the lag, and implying it’s therefore bad, when eventual consistency can include unnoticeably small lag. And thus it’s the lag that they’re actually bothered by, not the EC per se. Hence my clawed out hair.