5 ms·
Keeping an always consistent state in a large distributed you are fighting against the law of physics. Like it is mentioned, Google did it with their F1/Spanne
by rdtsc 12y ago
Keeping an always consistent state in a large distributed you are fighting against the law of physics.
Like it is mentioned, Google did it with their F1/Spanner SQL database. But that also mean GPS receivers with antennas on the roofs of data centers.
Which is yet another thing that can fail and it, either by itself or in a cascade of other failures will lead to unspecified and possibly undesirable behavior.
Recently I see a lot of advocates of dropping NoSQL databases and moving back to Postgress or other SQL databases.
The problem is SQL and schemas is not the only reason NoSQL databases became popular, they became popular because they also started to have a default and more defined behavior in respect to replication and distribution.
Most solutions don't need that and sticking with a solid single database works very well. But those that need distributed operation have a pretty hard task ahead of them.
One heuristic you can look at is if and how is distribution implemented. Is it something that is bolted on top, like you download some proxy or addon, or added as an afterthought, be very careful. Those things should be baked into the core. For example, does it support CRTDs? Does it have master to master replication and so on. If it claims to have consistency instead availability figure out how it is implemented, Paxos, Raft, or something else.
So far I think Riak is probably the database that thought the hardest and did the most reasonable job. The other simpler database is CouchDB, they have well specified conflict resolution behavior and master to master replication. But you have to usually write your own cluster topology. There are probably others but those are the two I know of first hand.
- manigandham 12y agoNice points, have to mention Aerospike as another that did a lot of the right stuff with regard to distributed actions and clustering.
- logicallee 12y agoI'm not an expert, and it sounds like you are, so I appreciate your feedback here: what do you even mean by a consistent state? even in theory a person initiating a new additional record in Auckland, New Zealand at the same time somebody iniatiates a change in Gibraltar or London (which are antipodal to the former[1]) 66 milliseconds away, cannot have a confirmation in less than 120 milliseconds, right? So do you just wait for that before declaring 'consistency'? Do you literally add 120 milliseconds to each and every request? (And this is assuming you have a damned good solution to the two generals problem)? I mean suppose the database tracks something as simple as: number of web page hits. It's a counter. You now distribute it, and have a stochastic process of counter hits between 0 and 5 per second in your largest cities, distributed throughout the world. How can that database ever be consistent? If there are ten new records per second in New Zealand and ten records per second in the UK, and they potentially depend on each other in some way, are you going to just make everyone wait until everything has been committed and confirmed to be consistent? Or is "a foolish consistency the hobgoblin of little minds", and you really can accept out-of-date data and deal with merging conflicts later? I just don't understand why we would expect consistency to rank up there, when we deal with a worldwide real-time system where the difference between getting served by a local database in 40 milliesconds and one far away in 250 milliseconds is both staggering, and incredibly noticeable. why be consistent? what is consistence? [1] http://www.findlatitudeandlongitude.com/antipode-map/ http://www.findlatitudeandlongitude.com/antipode-map/
- bdamm 12y agoThere are many systems which can work just fine in an eventually-consistent manner. A database of people (customers, users, etc) is a classic example of such a thing. In general I think consistency is over valued. There are plenty of cases where it is important. Lots of people are brainwashed in college to think that all data must be consistent all the time, and that's just not necessary.
- logicallee 12y ago>Lots of people are brainwashed in college to think that all data must be consistent all the time, and that's just not necessary. I knew it! So a foolish consistency is the hobgoblin of little minds.
- olau 12y agoWhile I agree strict consistency is probably overkill in many if not most situations, the problem with not having consistency is that it potentially makes the application logic much more complicated. Take the database of customers - so if you don't have consistency, what happens in case someone changes the company address and another person simultaneously requests a delivery of something. Do you risk ending up with half of the old address and half of the new one on the parcel? Note you can certainly have this problem in a consistent system too, e.g. if you make a UI without a save button where the address is changed one field at a time. Concurrency is just intrinsically hard.
- chousuke 12y ago"Lack of consistency" here doesn't have to imply lack of atomicity; In the case of your example, an eventually consistent system could return either the old address or the new address at some point in time X, but never a mixture of both. In your situation, the parcel could end up getting delivered to the old address, but there would be no corruption.
- nostrademons 12y agoNote that the real world operates like this too: before computers, if someone changes their address and simultaneously sends a package, the package probably will end up at the wrong address. We have a number of mechanisms in place to mitigate this when it occurs (address forwarding, return-to-sender, customer support, credit card chargebacks), but they still don't always work, and sometimes packages just get lost. The real world solution to this is the acceptance that yes, sometimes bad things happen for no reason at all. I suspect that the computer world will eventually move to this as well, with consumers becoming more tolerant of machines that simply give the wrong answer some of the time, as long as they give the wrong answer less frequently as a human would.
- threeseed 12y ago> Recently I see a lot of advocates of dropping NoSQL databases and moving back to Postgress or other SQL databases. This is just from a vocal minority on HN. You just need to look at the facts. Companies like Mongo, Datastax, Aerospike etc are growing bigger by the day, with increasingly higher valuations. Old school database companies like Teradata are now all about datalakes incorporating Hadoop and Mongo. And technologies like Spark, Impala are now on the front line for many data analytics and processing work. In the enterprise at least SQL databases are increasingly being relegated to a small part of the whole data pipeline i.e. storing the consolidated, integrated data model.
- pilsetnieks 12y agoThe advocates of dropping NoSQL and returning to SQL databases are indeed a minority — as the majority never went away from SQL databases, especially for the purposes they fulfil well. Right tool for the job and all that stuff.
- threeseed 12y agoAgain the facts simply don't agree with you. Companies have been moving away from SQL databases in droves compared to the 1990s. And why wouldn't they ? Vendors like Oracle, Microsoft, IBM etc have been screwing them over for ages. Low cost data lakes are the new norm. Of course I am talking about middle to enterprise companies. I am sure for individuals a typical LAMP setup is still going to be fine. But then again many of those are running their apps in the cloud and hence want a database that is resilient in the face of node outages. MySQL and PostgreSQL are both a PITA to get this right compared to almost every NoSQL database.
- louthy 12y ago> Again the facts simply don't agree with you. Companies have been moving away from SQL databases in droves compared to the 1990s. Do you have any links to said 'facts'? Otherwise your comments are just hearsay and anecdote.