5 ms·

SQL wins because big iron wins. At the end of the day, the big, clunky, slow, complicated, aggravating, old systems stick around because they are built to stick

SQL wins because big iron wins. At the end of the day, the big, clunky, slow, complicated, aggravating, old systems stick around because they are built to stick around. Looking at, say, an eventually consistent distributed decentralized kv store, one might be tricked into believing it's simple enough to deploy with enough nodes and general enough interfaces that you can build a complicated system on top of it if needed, and rely on its supppsed simplicity and scalability the rest of the time. But nobody tells you about the shitty implementation. The replication that doesn't resume, the stodgy transfers, the imbalanced distribution, the consensus conflict, the infinitely expanding disk, the churning CPU. How at scale, if all the other aspects of your infra aren't scaling along with your data size, the whole thing blows. Traditional SQL databases end up being many times simpler in practice, and because of their inherent scaling limitations, much easier to manage. And most importantly: their implementations aren't buggy. SQL is just more reliable.

wjossey 9y ago
As someone who helped design, then manage a massive NoSQL store which hundreds of terabytes of data, of which most of it was hot, I’ll humbly disagree and say it’s not necessarily as bad as you make it sound. Did we have someone who managed that system? Sometimes, but mostly it just did it’s own thing. We ironically invested way more time on our MySQL database over the years because we couldn’t get that to scale the way we wanted to, but I think that was specific to a problem we were having. Did we later invest a lot of time in that system (the NoSQL one)? Yes, because it was very cost effective for us to do so. At certain scale throwing people at optimization problems can pay huge dividends. But, this can be said of most infrstracture. It’s usually worth revisiting every year or two and seeing how what you can squeeze out. Did we have churning CPU, infinitely expanding disk, replication issues, and more? Sure, but not very commonly and mostly it was fairly easily resolved. More importantly though, it was a solid system that was the underpinning of a colossal system, and it behaved admiraly more than 99.9% of the time. Will most projects benefit from a hugely distributed KV store? Nope. But I’m still glad they exist!
- cachemiss 9y ago
  Caveat, this comment isn't directed at you (I agree with your comment), but rather the points around what you are saying. One thing that helps is if people stop referring to things as SQL / NoSQL as what ends up happening is various things get conflated. When talking about stores, it's important to be explicit about a few things: 1. Storage model 2. Distribution model 3. Access model 4. Transaction model 5. Maturity and competence of implementation What happens is people talk about "SQL" as either an NSM or DSM storage model, over either a single node, or possibly more than that in some of the MPP systems, using SQL as an access model, with linearizable transactions, and a mature competent implementation. NoSQL when most people refer to it can be any combination of those things, as long as the access model isn't SQL. I work on database engines, and it's important to decouple these things and be explicit about them when discussing various tradeoffs. You can do SQL the language over a distributed k/v store (not always a great idea) and other non-tabular / relational models and you can distribute relational engines (though scaling linearizable transactions is difficult and doesn't scale for certain use cases due to physics, but that's unrelated to the relational part of it). Generally people talk about joins not scaling in some normalized form, but then what they do is just materialize the join into whatever they are using to store things in a denormalized model, which has its own drawbacks. As to the comment above you, SQL vs NoSQL also doesn't have anything to do with the relative maturity of anything. Some of the newer non-relational engines have some operational issues, but that doesn't really have anything to do with their storage model or access method, it just has to due with the competence of the implementation. MongoDB is difficult operationally not because it's not a relational engine, but because it wasn't well designed. Just like people put SQL over non-tabular stores, you can build non-tabular / relational engines over relational engines (sharding PostgreSQL etc.). In fact major cloud vendors do just that.
  - wjossey 9y ago
    Wonderful response. Thank you. Wish I could give multiple upvotes. I’ll add some of those points to my thought process going forward.
jjirsa 9y ago
There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead and describe resharding a sql DB as your company grows 100x Or how you’ll handle a dc failover and fail back Or how you’ll shrink your resources after the holidays - without downtime. These are trivial with Cassandra and god damn near impossible for most MySQL users. More reliable? Your “big iron” is still a single point of failure. If not a single cpu, a raid card or a top of rack switch or a DC in a hurricane zone.
- elvinyung 9y ago
  You got downvoted, but I agree with you. Traditional RDBMS is inherently single-node and a pain to scale. The actual sad thing here is that there's no mature/reliable commodity (i.e. open source) distributed RDBMS for transaction processing yet.
  - yenwel 9y ago
    VoltDB?
  - rhencke 9y ago
    CockroachDB? I'll grant you it's still young, but it pretty solidly hits all your other checkboxes.
    - elvinyung 9y ago
      IMO, it's very promising but still extremely immature. I still believe in Curt Monash's cardinal rule of RDBMS development: http://www.dbms2.com/2013/03/18/dbms-development-marklogic-hadoop/ http://www.dbms2.com/2013/03/18/dbms-development-marklogic-h...
  - jinqueeny 9y ago
    I have to bring this up again: NewSQL. There are quite a few new choices out there, Google Spanner, Cockroachdb(https://github.com/cockroachdb/cockroach https://github.com/cockroachdb/cockroach), TiDB (https://github.com/pingcap/tidb https://github.com/pingcap/tidb). All of them are very easy to scale while maintaining the ACID transactions.
imtringued 9y ago
SQL is just a querying language, the point of the article is that it's not exclusive to RDBMS, it is also used by NoSQL databases. There is also no law that says RDBMS must be strongly consistent. They can also be eventually consistent. The opposite also applies to NoSQL databases.

wjossey 9y ago

As someone who helped design, then manage a massive NoSQL store which hundreds of terabytes of data, of which most of it was hot, I’ll humbly disagree and say it’s not necessarily as bad as you make it sound. Did we have someone who managed that system? Sometimes, but mostly it just did it’s own thing. We ironically invested way more time on our MySQL database over the years because we couldn’t get that to scale the way we wanted to, but I think that was specific to a problem we were having. Did we later invest a lot of time in that system (the NoSQL one)? Yes, because it was very cost effective for us to do so. At certain scale throwing people at optimization problems can pay huge dividends. But, this can be said of most infrstracture. It’s usually worth revisiting every year or two and seeing how what you can squeeze out. Did we have churning CPU, infinitely expanding disk, replication issues, and more? Sure, but not very commonly and mostly it was fairly easily resolved. More importantly though, it was a solid system that was the underpinning of a colossal system, and it behaved admiraly more than 99.9% of the time. Will most projects benefit from a hugely distributed KV store? Nope. But I’m still glad they exist!

cachemiss 9y ago
Caveat, this comment isn't directed at you (I agree with your comment), but rather the points around what you are saying. One thing that helps is if people stop referring to things as SQL / NoSQL as what ends up happening is various things get conflated. When talking about stores, it's important to be explicit about a few things: 1. Storage model 2. Distribution model 3. Access model 4. Transaction model 5. Maturity and competence of implementation What happens is people talk about "SQL" as either an NSM or DSM storage model, over either a single node, or possibly more than that in some of the MPP systems, using SQL as an access model, with linearizable transactions, and a mature competent implementation. NoSQL when most people refer to it can be any combination of those things, as long as the access model isn't SQL. I work on database engines, and it's important to decouple these things and be explicit about them when discussing various tradeoffs. You can do SQL the language over a distributed k/v store (not always a great idea) and other non-tabular / relational models and you can distribute relational engines (though scaling linearizable transactions is difficult and doesn't scale for certain use cases due to physics, but that's unrelated to the relational part of it). Generally people talk about joins not scaling in some normalized form, but then what they do is just materialize the join into whatever they are using to store things in a denormalized model, which has its own drawbacks. As to the comment above you, SQL vs NoSQL also doesn't have anything to do with the relative maturity of anything. Some of the newer non-relational engines have some operational issues, but that doesn't really have anything to do with their storage model or access method, it just has to due with the competence of the implementation. MongoDB is difficult operationally not because it's not a relational engine, but because it wasn't well designed. Just like people put SQL over non-tabular stores, you can build non-tabular / relational engines over relational engines (sharding PostgreSQL etc.). In fact major cloud vendors do just that.
- wjossey 9y ago
  Wonderful response. Thank you. Wish I could give multiple upvotes. I’ll add some of those points to my thought process going forward.

jjirsa 9y ago

There is so much wrong with this post I don’t know where to start Old school sql replication is awful. It’s awful in Postgres (single threaded replay? A single vacuum can make your slaves fall behind). It’s awful in MySQL (go ahead and tell me how you do dual master without rewriting your app). People use nosql not because it’s eady or flawless, but because doing it with sql is an order of magnitude harder. Go ahead and describe resharding a sql DB as your company grows 100x Or how you’ll handle a dc failover and fail back Or how you’ll shrink your resources after the holidays - without downtime. These are trivial with Cassandra and god damn near impossible for most MySQL users. More reliable? Your “big iron” is still a single point of failure. If not a single cpu, a raid card or a top of rack switch or a DC in a hurricane zone.

elvinyung 9y ago
You got downvoted, but I agree with you. Traditional RDBMS is inherently single-node and a pain to scale. The actual sad thing here is that there's no mature/reliable commodity (i.e. open source) distributed RDBMS for transaction processing yet.
- yenwel 9y ago
  VoltDB?
- rhencke 9y ago
  CockroachDB? I'll grant you it's still young, but it pretty solidly hits all your other checkboxes.
  - elvinyung 9y ago
    IMO, it's very promising but still extremely immature. I still believe in Curt Monash's cardinal rule of RDBMS development: http://www.dbms2.com/2013/03/18/dbms-development-marklogic-hadoop/ http://www.dbms2.com/2013/03/18/dbms-development-marklogic-h...
- jinqueeny 9y ago
  I have to bring this up again: NewSQL. There are quite a few new choices out there, Google Spanner, Cockroachdb(https://github.com/cockroachdb/cockroach https://github.com/cockroachdb/cockroach), TiDB (https://github.com/pingcap/tidb https://github.com/pingcap/tidb). All of them are very easy to scale while maintaining the ACID transactions.

imtringued 9y ago

SQL is just a querying language, the point of the article is that it's not exclusive to RDBMS, it is also used by NoSQL databases. There is also no law that says RDBMS must be strongly consistent. They can also be eventually consistent. The opposite also applies to NoSQL databases.