5 ms·

How did SQL get so popular?

15 points by mattmichielsen 17y ago

silentbicycle 17y ago
The post's argument appears to be that SQL (by which he means relational databases, as noted in the comments) are popular because implementing ACID is too much trouble, and because dealing with random file access is confusing. It sounds like it never even crossed his mind that the relational model is a good fit for some problems. Wow.
- davidmathers 17y ago
  It sounds like it never even crossed his mind that the relational model is a good fit for some problems. 98% of programmers think the word "relation" has something to do with links between tables. I doubt the OP has ever even heard of a concept called "the relational model".
  - silentbicycle 17y ago
    Touché.
StrawberryFrog 17y ago
My first thought was "Popular compared to what? What was the alternative?" I was expecting to see hierarchical databases, OODBs and alternate query languages mentioned. The claim for "Distaste" for SQL is odd and vague. I agree with the first point - "Relational databases are ACID and that's powerful" but I ask "so what, where's the distaste?" People who know a bit about RDBMSs know that they do a lot of heavy lifting for you. Is the point that a younger generation don't know that? What are they teaching kids in college these days? Massively parallel data storage systems are in vogue now, bacause of the very the few very popular web apps that need to scale a lot horizontally, and SQL databases don't really deal with that. But that's not "How SQL got popular". Maybe it's why some have forgotten.
- silentbicycle 17y ago
  Whenever I read articles like this, I get the impression that an apparently sizable (or just vocal?) subset of the people who "know SQL" learned it from some sort of "Teach Yourself PHP & MySQL in 24 Hours" book, and completely missed learning about how the relational model actually works, how to normalize a table, etc. They have a terrible experience using MySQL on a database with one huge, un-indexed table, and they're convinced for life that all RDBMSs are crap. I mean, if you don't like relational databases for a real reason, that's one thing, but this is like complaining because gcc won't run a Visual Basic program you copy-and-pasted, and having half the people you whine to not know any better, so they tell their friends too. I can understand when people complain that SQL is a lousy implementation of the relational model (or about its syntax being a bit verbose, etc.), but that's clearly not the case here.
- prpon 17y ago
  I agree with you here. Is there any one here on HN who is not comfortable with SQL? However, the increasing usage of frameworks such as rails/grails/hibernate/JPA makes it easier to access data without having to worry about SQL until something breaks. The prime time for SQL has long been over, it is a necessary tool most of us don't think too much about.
  - timwiseman 17y ago
    The prime time for SQL has long been over, it is a necessary tool most of us don't think too much about. -- I have to respectfully disagree. Its often when a tool becomes so ubiquitious and sufficiently intuitive that its true power comes to the fore. It is like the electric grid, when it was first being forged everyone who used it thought about it extensively, now hardly anyone does. But that does not mean the grid's prime time is over, it means that now it is truly becoming powerful.
  - silentbicycle 17y ago
    Relational databases are designed for dealing with huge collections of data. Huge like terabytes. (This is, of course, their very reason for existence.) Every time a framework pulls a big set of data across the wire to do a local calculation on it (a for loop summing a list of values, for example), it's doing a completely unnecessary round trip to do (probably in PHP or Ruby, no less) a task for which the database has already been specifically optimized. It could have done all the calculation during the same pass it took to select the records and just sent the answer instead. Frameworks obscure this. Just a thought.
seldo 17y ago
A more interesting question to me is "how did SQL get so unpopular all of a sudden?" It seems to me it went like this: 1. modern programming frameworks all abstracted away SQL because people found it hard to understand 2. SQL written by ORM layers in these frameworks is slow. So people decided this was because RDBMS are slow. 3. So people decided to stop using RDBMS and use object-oriented data stores instead (which is faster, unless you have relational queries, which a large number of simple web applications do not) 4. Then when they started needing relational queries again, they started building complicated custom layers on top of their object stores to handle this I figure sometime in the future somebody will publish a "brilliant" article called "Save yourself thousands of lines of object-manipulation code... by using RDBMS!" and suddenly SQL will be "popular" again. I'm the first to admit that the default approach to web development for the past 10 years has been "slap all your data into mySQL, then start writing code". It works in a very, very large number of cases, but it's not super-efficient. It never claimed to be. It was just really easy and quick to write. Now tools are growing up that let you solve these sorts of problems without using RDBMS, and that's great: the huge number of apps that don't need RDBMS can stop using SQL, and let us SQL junkies get on with our lives :-)
- dagheti 17y ago
  I think part of the backlash is coming from developers on web-applications that have business models that require scaling to very large loads before they make sense. This puts a premium on keeping things very cheap (no heavy commercial SQLDBMS), and inherently scalable (hierarchical). Web developers on these types of projects cannot either a: buy a "real" SQLDBMS, or b: charge their customers so their business model makes sense without a huge number of transactions. This means they are willing to suffer it out with solutions that don't provide flexibility or data correctness guarantees of a RDBMS. This is just a guess, but it's quite likely that it is the rise of hyper-scale-intensive, speed-intensive, revenue-light business models that drives devs towards deciding SQL isn't worth it.
  - silentbicycle 17y ago
    Postgresql is free. I'm not going to claim it's perfect, but it's pretty damn good, and postgres + intelligently used caching (such as memcached) goes a long way.
wglb 17y ago
There seems to be some confusion between SQL, ACID(Atomicity, Consistency, Isolation, Durability), and a RDBMS. And it would have been nice, as StrawberryFrog notes to see these concepts compared to OODB and other query languages. Missing is the concept that SQL is really quite powerful.
simc 17y ago
SQL is popular because it was the query language that was available in the Oracle and IBM relational databases, which were the first relational database products to come to market. SQL subsequently became an ISO standard. Eventually all relational database vendors switched to SQL. Other DBMSs (Ingres, Informix, Postgres) originally used QUEL which was thought by many to be superior to SQL eventually switched as SQL became dominant/the standard. (Although QUEL can still be used with Ingres). There may be other reasons, but I think that in large part these are the reasons why SQL is dominant today. Relational databases became popular because the declarative high level query language was less time consuming to write queries in than earlier non-relational database. SQL is far from the perfect query language though. It's syntax is often bizarre and confounding, and has many non relational extensions (column ordering, duplicate tuples in a relation).
- catch404 17y ago
  This is the real answer to the question.
- azgolfer 17y ago
  Re: non relational extensions I believe it was Ted Codd who said "No real application runs in third normal form". The relational model is great, and it's important to remember that the things that make it great are that it is simple, dynamic, easy to understand and helps ensure you get correct results. IMO the two most common mistakes people make are being overly worried about breaking the model where necessary, and conversely, not realizing the benefits of things like human readable keys.
bayareaguy 17y ago
SQL didn't really become popular until companies started believing in the benefits of moving their applications from homogeneous mainframe and timesharing systems to distributed heterogeneous client/server networks.

The post's argument appears to be that SQL (by which he means relational databases, as noted in the comments) are popular because implementing ACID is too much trouble, and because dealing with random file access is confusing. It sounds like it never even crossed his mind that the relational model is a good fit for some problems. Wow.

davidmathers 17y ago
It sounds like it never even crossed his mind that the relational model is a good fit for some problems. 98% of programmers think the word "relation" has something to do with links between tables. I doubt the OP has ever even heard of a concept called "the relational model".
- silentbicycle 17y ago
  Touché.

StrawberryFrog 17y ago

My first thought was "Popular compared to what? What was the alternative?" I was expecting to see hierarchical databases, OODBs and alternate query languages mentioned. The claim for "Distaste" for SQL is odd and vague. I agree with the first point - "Relational databases are ACID and that's powerful" but I ask "so what, where's the distaste?" People who know a bit about RDBMSs know that they do a lot of heavy lifting for you. Is the point that a younger generation don't know that? What are they teaching kids in college these days? Massively parallel data storage systems are in vogue now, bacause of the very the few very popular web apps that need to scale a lot horizontally, and SQL databases don't really deal with that. But that's not "How SQL got popular". Maybe it's why some have forgotten.

silentbicycle 17y ago
Whenever I read articles like this, I get the impression that an apparently sizable (or just vocal?) subset of the people who "know SQL" learned it from some sort of "Teach Yourself PHP & MySQL in 24 Hours" book, and completely missed learning about how the relational model actually works, how to normalize a table, etc. They have a terrible experience using MySQL on a database with one huge, un-indexed table, and they're convinced for life that all RDBMSs are crap. I mean, if you don't like relational databases for a real reason, that's one thing, but this is like complaining because gcc won't run a Visual Basic program you copy-and-pasted, and having half the people you whine to not know any better, so they tell their friends too. I can understand when people complain that SQL is a lousy implementation of the relational model (or about its syntax being a bit verbose, etc.), but that's clearly not the case here.
prpon 17y ago
I agree with you here. Is there any one here on HN who is not comfortable with SQL? However, the increasing usage of frameworks such as rails/grails/hibernate/JPA makes it easier to access data without having to worry about SQL until something breaks. The prime time for SQL has long been over, it is a necessary tool most of us don't think too much about.
- timwiseman 17y ago
  The prime time for SQL has long been over, it is a necessary tool most of us don't think too much about. -- I have to respectfully disagree. Its often when a tool becomes so ubiquitious and sufficiently intuitive that its true power comes to the fore. It is like the electric grid, when it was first being forged everyone who used it thought about it extensively, now hardly anyone does. But that does not mean the grid's prime time is over, it means that now it is truly becoming powerful.
- silentbicycle 17y ago
  Relational databases are designed for dealing with huge collections of data. Huge like terabytes. (This is, of course, their very reason for existence.) Every time a framework pulls a big set of data across the wire to do a local calculation on it (a for loop summing a list of values, for example), it's doing a completely unnecessary round trip to do (probably in PHP or Ruby, no less) a task for which the database has already been specifically optimized. It could have done all the calculation during the same pass it took to select the records and just sent the answer instead. Frameworks obscure this. Just a thought.

seldo 17y ago

A more interesting question to me is "how did SQL get so unpopular all of a sudden?" It seems to me it went like this: 1. modern programming frameworks all abstracted away SQL because people found it hard to understand 2. SQL written by ORM layers in these frameworks is slow. So people decided this was because RDBMS are slow. 3. So people decided to stop using RDBMS and use object-oriented data stores instead (which is faster, unless you have relational queries, which a large number of simple web applications do not) 4. Then when they started needing relational queries again, they started building complicated custom layers on top of their object stores to handle this I figure sometime in the future somebody will publish a "brilliant" article called "Save yourself thousands of lines of object-manipulation code... by using RDBMS!" and suddenly SQL will be "popular" again. I'm the first to admit that the default approach to web development for the past 10 years has been "slap all your data into mySQL, then start writing code". It works in a very, very large number of cases, but it's not super-efficient. It never claimed to be. It was just really easy and quick to write. Now tools are growing up that let you solve these sorts of problems without using RDBMS, and that's great: the huge number of apps that don't need RDBMS can stop using SQL, and let us SQL junkies get on with our lives :-)

dagheti 17y ago
I think part of the backlash is coming from developers on web-applications that have business models that require scaling to very large loads before they make sense. This puts a premium on keeping things very cheap (no heavy commercial SQLDBMS), and inherently scalable (hierarchical). Web developers on these types of projects cannot either a: buy a "real" SQLDBMS, or b: charge their customers so their business model makes sense without a huge number of transactions. This means they are willing to suffer it out with solutions that don't provide flexibility or data correctness guarantees of a RDBMS. This is just a guess, but it's quite likely that it is the rise of hyper-scale-intensive, speed-intensive, revenue-light business models that drives devs towards deciding SQL isn't worth it.
- silentbicycle 17y ago
  Postgresql is free. I'm not going to claim it's perfect, but it's pretty damn good, and postgres + intelligently used caching (such as memcached) goes a long way.

wglb 17y ago

There seems to be some confusion between SQL, ACID(Atomicity, Consistency, Isolation, Durability), and a RDBMS. And it would have been nice, as StrawberryFrog notes to see these concepts compared to OODB and other query languages. Missing is the concept that SQL is really quite powerful.

simc 17y ago

SQL is popular because it was the query language that was available in the Oracle and IBM relational databases, which were the first relational database products to come to market. SQL subsequently became an ISO standard. Eventually all relational database vendors switched to SQL. Other DBMSs (Ingres, Informix, Postgres) originally used QUEL which was thought by many to be superior to SQL eventually switched as SQL became dominant/the standard. (Although QUEL can still be used with Ingres). There may be other reasons, but I think that in large part these are the reasons why SQL is dominant today. Relational databases became popular because the declarative high level query language was less time consuming to write queries in than earlier non-relational database. SQL is far from the perfect query language though. It's syntax is often bizarre and confounding, and has many non relational extensions (column ordering, duplicate tuples in a relation).

catch404 17y ago
This is the real answer to the question.
azgolfer 17y ago
Re: non relational extensions I believe it was Ted Codd who said "No real application runs in third normal form". The relational model is great, and it's important to remember that the things that make it great are that it is simple, dynamic, easy to understand and helps ensure you get correct results. IMO the two most common mistakes people make are being overly worried about breaking the model where necessary, and conversely, not realizing the benefits of things like human readable keys.

bayareaguy 17y ago

SQL didn't really become popular until companies started believing in the benefits of moving their applications from homogeneous mainframe and timesharing systems to distributed heterogeneous client/server networks.