3 ms·
> It is not about that Relational (SQL) database was designed in a way that is hard to make it distributed Not entirely accurate. Relational database schemas a
by lobster_johnson 11y ago
> It is not about that Relational (SQL) database was designed in a way that is hard to make it distributed
Not entirely accurate. Relational database schemas are typically normalized into many tables and make heavy use of joins; it's fair to say that the concept of joins are integral the relational model. But joins, by definition, require fast access to data; it's extremely inefficient to join across network boundaries.
So-called NoSQL databases aren't just about ACID or CAP, but also about denormalization. In fact, I would go so far as to say that the single founding tenet of all of "NoSQL" was denormalization across multiple nodes. Google's Bigtable (on which HBase is based, and which arguably started the public's interest in "NoSQL") depends on extreme denormalization in its data model‚ which does away with tables and instead treats everything as columns, and also discards referential integrity (the "C" in ACID, which isn't the same as in CAP) and isolation (the "I"). Sharding flows naturally out of denormalization. Eventual consistency came later with Amazon's Dynamo paper; Bigtable was strongly consistent.
A consequence of denormalization is that you can hit any database node and get consistent performance. No query will ever get bogged down in complicated, long-running nested loops and hash joins because the mechanisms don't exist. It creates an even playing field for queries because they're mostly individual key lookups; the result is (in theory) predictable, stable performance across the cluster. The complexity of join logic that was in the relational database is moved into the client, which gets the burden of untangling data. Which is why Google also invented early solutions like MapReduce and Sawzall, because once you have a huge, denormalized, distributed cluster of data in this poor man's relational model, your individual clients can no longer access the data efficiently. So denormalization also leads to parallel computation, and especially precomputation, rather than the ad-hoc queries that relational databases are good at.
It's all about scale, and fitting the data model to work in your favour at that scale.
So yeah, I would definitely say that relational databases are designed in a way that make them hard to distribute. The above is just the tip of the iceberg. How do you enforce referential integrity (foreign keys, etc.) across nodes? How do you assign unique primary keys? You talk about making tradeoffs, but would you really want a distributed relational database without distributed transactions?
Can you implement a performant distributed database with joins and other relational features? Yes. VoltDB and Vertica come to mind. They are designed from the ground up to do this, and have complicated logic for partitioning queries. They make other tradeoffs: for example, VoltDB is in-memory and to perform well, all transactional access code has to be written as a "stored procedure", a Java JAR file that you upload to the server, and transactions are expected to be very short-lived. But they will, pretty much by definition, not perform at scale the way that "NoSQL" databases are designed to.
- manigandham 11y agoTake a look at MemSQL - a much better distributed relational database with regular mysql protocol access.
- lobster_johnson 11y agoThanks, but I'm not going to consider a database that's not open source, sorry.