4 ms·

That's... still a distributed system with all the joys of eventual consistency.

jdc 9y ago
However, I do think the general consensus is that stateless/shared-nothing app servers and following something like the 12 Factor App[1] pattern makes things much easier. 1. https://12factor.net https://12factor.net
- saryant 9y ago
  Yes, it does. But just because your storage layer is abstracted behind an API doesn't mean eventual consistency isn't your concern. Take S3 for example. LIST calls lag deletes but exists() doesn't, so exists() could return 404 for a document that's still included in the result of LIST. Plus, all exists() calls go through a caching load-balancer, so you could receive different answers in two subsequent exists() calls. That's just a small subset. There's a good comment here[1] on S3's consistency semantics. Point is, 12FA makes things easier but any time you have more than one computer involved (i.e, always), consistency becomes complicated. [1] https://issues.apache.org/jira/browse/FLINK-5706 https://issues.apache.org/jira/browse/FLINK-5706
  - zip1234 9y ago
    Seems like when cache needs to get involved, things can get complicated very quickly.
    - dmoy 9y ago
      One of the two hard problems...
  - hamandcheese 9y ago
    I don't really see how your example applies to the point the root comment made. S3 surely _is_ a distributed system, but he is specifically talking about data residing on _one_ machine (notwithstanding the bits about read only followers).
hueving 9y ago
Only if the db replicas allow stale reads.
- saryant 9y ago
  A replica always lags master by some degree. It may be small—until things go sideways—but it's still another form of eventual consistency.
  - peterwwillis 9y ago
    You're saying ACID databases are eventually consistent?
    - saryant 9y ago
      I'm saying read replicas are not instantly in sync with their masters. Updates are asynchronously copied to the replica.
      - peterwwillis 9y ago
        Or, you know, use synchronous replication, or semi-synchronous, or a different asynchronous model that isn't weakly consistent such as distributed transaction logs, write-read consensus, complicated locking mechanisms, etc
        saryant 9y ago
        All of these solutions are still battling the fact that distributed systems are hard and there's a tradeoff to every decision. If you use synchronous replication, you now require at least two systems to commit a write, so you've increased the chance that your commit will fail. Maybe that's the right choice—but it's a tradeoff and the decision ultimately rests in your business requirements. My point is simply that there is no magic bullet in ensuring that all relevant systems have the same answer to the same question at the same time.
di4na 9y ago
Especially on the load balancer side...

jdc 9y ago

However, I do think the general consensus is that stateless/shared-nothing app servers and following something like the 12 Factor App[1] pattern makes things much easier. 1. https://12factor.net https://12factor.net

saryant 9y ago
Yes, it does. But just because your storage layer is abstracted behind an API doesn't mean eventual consistency isn't your concern. Take S3 for example. LIST calls lag deletes but exists() doesn't, so exists() could return 404 for a document that's still included in the result of LIST. Plus, all exists() calls go through a caching load-balancer, so you could receive different answers in two subsequent exists() calls. That's just a small subset. There's a good comment here[1] on S3's consistency semantics. Point is, 12FA makes things easier but any time you have more than one computer involved (i.e, always), consistency becomes complicated. [1] https://issues.apache.org/jira/browse/FLINK-5706 https://issues.apache.org/jira/browse/FLINK-5706
- zip1234 9y ago
  Seems like when cache needs to get involved, things can get complicated very quickly.
  - dmoy 9y ago
    One of the two hard problems...
- hamandcheese 9y ago
  I don't really see how your example applies to the point the root comment made. S3 surely _is_ a distributed system, but he is specifically talking about data residing on _one_ machine (notwithstanding the bits about read only followers).

hueving 9y ago

Only if the db replicas allow stale reads.

saryant 9y ago
A replica always lags master by some degree. It may be small—until things go sideways—but it's still another form of eventual consistency.
- peterwwillis 9y ago
  You're saying ACID databases are eventually consistent?
  - saryant 9y ago
    I'm saying read replicas are not instantly in sync with their masters. Updates are asynchronously copied to the replica.
    - peterwwillis 9y ago
      Or, you know, use synchronous replication, or semi-synchronous, or a different asynchronous model that isn't weakly consistent such as distributed transaction logs, write-read consensus, complicated locking mechanisms, etc
      - saryant 9y ago
        All of these solutions are still battling the fact that distributed systems are hard and there's a tradeoff to every decision. If you use synchronous replication, you now require at least two systems to commit a write, so you've increased the chance that your commit will fail. Maybe that's the right choice—but it's a tradeoff and the decision ultimately rests in your business requirements. My point is simply that there is no magic bullet in ensuring that all relevant systems have the same answer to the same question at the same time.

di4na 9y ago

Especially on the load balancer side...