3 ms·
> Building the complex, scalable systems demanded by todays highly connected world with such weak guarantees is exceptionally difficult. So is building a compl
by rdtsc 12y ago
> Building the complex, scalable systems demanded by todays highly connected world with such weak guarantees is exceptionally difficult.
So is building a complex scalable system that break the laws of physics or theorems. The world is eventually consistent. Some business domains can handle that. Even some banking operations are eventually consistent. You can get to an ATM in Australia and one in US at the same time and overdraw your account. That is eventually consistency. Banks see it better that way then expect you to wait for half an hour until they can decide on a global shared consistent state of your account.
> Essentially, that engineer needs to manually do the hard work to ensure that multiple clients don’t step on each other’s toes and deal with stale data.
Sometimes that is not that hard. Sometimes business logic allows for a custom (user-based) reconciliation of conflicts. In some cases that engineer has to rely on magic unicorns that another engineer (who build the DB) put it in the product to make it beat the CAP theorem. Or the administrator needs to handle global restart of all the servers because the global cluster has become unavailable because say one node has blown up or got partitioned. That is not _always_ in all cases better than the case of eventual consistency.
> Google addressed the pain points of eventual consistency in a recent paper on its F1 database
So the answer is installing expensive GPS receiver on the roof of your data-centers and running wires down to the cluster of machines? Yeah that works better in some cases. But it is not the best answer always.
> Vendors should stop hiding behind the CAP theorem as a justification for eventual consistency.
Vendors should lie and market-speak their way out of a theorem? This has been done before with other database products. So maybe FoundationDB is choosing that path to follow...
> Dave Rosenthal is a co-founder of FoundationDB.
I don't know. As I often say, sometimes the biggest enemies of a an idea are its most ardent supporters.
- bane 12y ago> So is building a complex scalable system that break the laws of physics or theorems. The world is eventually consistent. You kind of hit me during a moment with this. You're basically describing the theory of relativity. I suppose in a way, the ToR could be rephrased to be a theoretical upper limit on information propagation with ramifications for ensuring consistency in local state.
- Dave_Rosenthal 12y ago"beat the CAP theorem" ... "break the laws of physics" ... "lie and market-speak their way out of a theorem" Strong words, but the article doesn't advocate any of the above. It advocates choosing consistency over availability and lays out the reasons why that is a good choice. Choosing consistency is hardly a radical idea and many of the most popular NoSQL databases actually choose consistency (e.g. HBase, MongoDB). Indeed even Riak, the biggest advocate of eventual consistency, is working hard to build strong consistency into Riak 2.0.
- rdtsc 12y agoBut choosing consistency over availability in all cases is not an answer, one is not a strictly superior choice over another, as he article proposes. At one level striving for consistency in a large distributed system is fighting with the laws of physics. It has been attempted and so far Google probably has a better handle on it but it requires tight coupling with a time synchronization service. > Choosing consistency is hardly a radical idea _Claiming_ to choose it is not radical. Actually doing it, is. If you read the "Call me maybe" you'd see that most of supposed consistent databases fail. Granted that measures partition tolerance but at the same time usually that is not a choice to be made (according to the author Aphyr). > Indeed even Riak, the biggest advocate of eventual consistency, is working hard to build strong consistency into Riak 2.0. That is what I've heard. Consistency for some data is the right choice, no argument there. At the same time they are _not_ throwing away or switching away from eventual consistency. In fact they are the database that when preserving conflicted siblings actually did better than most in "Call me maybe" series. CouchDB is another database that does this right. It preserves merge conflicts explicitly during replication. Users can chose to ignore them but it doesn't arbitrary throw data away. It is not in the series because replication and cluster topology is user defined and managed so there is no default, single distributed cluster setup. Most of the failures of databases come from trying to sweep under the rug effects of conflicts in eventually consistency. Even Riak's last-write-wins did that. MongoDB and others failed too. That doesn't mean eventual consistency is flawed. Eventual consistency is a physical reality, and in some case it is also a viable business application pattern.
- 12y ago
- mtdewcmu 12y ago>>has to rely on magic unicorns that another engineer (who build the DB) put it in the product Where can I get these?
- rdtsc 12y agoPick any failed cases from "Call Me Maybe" series. In general unicorns materialize better if you read only marketing materials and follow PR pieces in picking your databases. Then they tend to disappear when written data is silently dropped on the floor.