5 ms·

A more interesting question to me is "how did SQL get so unpopular all of a sudden?" It seems to me it went like this: 1. modern programming frameworks all abs

by seldo 17y ago

A more interesting question to me is "how did SQL get so unpopular all of a sudden?" It seems to me it went like this: 1. modern programming frameworks all abstracted away SQL because people found it hard to understand 2. SQL written by ORM layers in these frameworks is slow. So people decided this was because RDBMS are slow. 3. So people decided to stop using RDBMS and use object-oriented data stores instead (which is faster, unless you have relational queries, which a large number of simple web applications do not) 4. Then when they started needing relational queries again, they started building complicated custom layers on top of their object stores to handle this I figure sometime in the future somebody will publish a "brilliant" article called "Save yourself thousands of lines of object-manipulation code... by using RDBMS!" and suddenly SQL will be "popular" again. I'm the first to admit that the default approach to web development for the past 10 years has been "slap all your data into mySQL, then start writing code". It works in a very, very large number of cases, but it's not super-efficient. It never claimed to be. It was just really easy and quick to write. Now tools are growing up that let you solve these sorts of problems without using RDBMS, and that's great: the huge number of apps that don't need RDBMS can stop using SQL, and let us SQL junkies get on with our lives :-)

dagheti 17y ago
I think part of the backlash is coming from developers on web-applications that have business models that require scaling to very large loads before they make sense. This puts a premium on keeping things very cheap (no heavy commercial SQLDBMS), and inherently scalable (hierarchical). Web developers on these types of projects cannot either a: buy a "real" SQLDBMS, or b: charge their customers so their business model makes sense without a huge number of transactions. This means they are willing to suffer it out with solutions that don't provide flexibility or data correctness guarantees of a RDBMS. This is just a guess, but it's quite likely that it is the rise of hyper-scale-intensive, speed-intensive, revenue-light business models that drives devs towards deciding SQL isn't worth it.
- silentbicycle 17y ago
  Postgresql is free. I'm not going to claim it's perfect, but it's pretty damn good, and postgres + intelligently used caching (such as memcached) goes a long way.
  - kragen 17y ago
    Well, you're right that licensing cost can't be the reason, although ten years ago it could have been. But I think the "inherently scalable" item may have something to it.
TallGuyShort 17y ago
You're absolutely right. In my project, we were using a Java API to get some numbers from a huge database, but the calls took up way to many resources (time and memory). As an experiment, I tried querying the database directly in my app (bypassing the abstracted framework) and it was lightning fast. My code was hard to understand (and very hard to write, at times) but the result was a much lighter, much faster application.
- wlievens 17y ago
  So you need a framework that allows you to do both, without it falling apart.
xenophanes 17y ago
> 1. modern programming frameworks all abstracted away SQL because people found it hard to understand No. Not because it's complicated. Because they wanted higher level facilities for getting/setting data. In other words, they put in abstraction because abstraction is good, not because they are dumb.
- dave_au 17y ago
  There's probably two sets of people at work in the statement. The people who made the frameworks would require a good understand as well as the idea that an abstraction might make life easier. The side effect is that if you create a good enough abstraction you'll get people who can handle the abstraction but are a little shakey when dealing with what is abstracted. See, you can both be right :)
- seldo 17y ago
  Abstraction is good, but I think the problem is that ORM is not a good abstraction for anything but the trivial case of a table. As soon as you want to do anything actually relational, I have yet to see an ORM layer that comes up with an efficient join. The reason, say, C++ is a good abstraction for machine code is because it simplifies what's going on to the point that you can write something very complex in fewer lines of code, allowing it to fit into the human brain better. I'm willing to be shown examples to the contrary, but ORM layers don't simplify or compress relational queries, they just change the syntax to something more "object-y". So maybe my point is that abstraction is good, but ORM isn't really abstraction.
  - trezor 17y ago
    ORM is not a good abstraction for anything but the trivial case of a table. As soon as you want to do anything actually relational, I have yet to see an ORM layer that comes up with an efficient join. Microsoft's Linq for SQL generates mostly OK SQL from your code. While not perfect for all complex operations (like multiple joins with aggregations), the SQL isn't horrible. For most use cases though, the SQL seems to be on par with what I would write myself. Ofcourse this is a rather vendor-specific implementation (for Microsoft SQL Server), but there are decent ORMs out there. The biggest problem really comes when people don't understand what underlying SQL will be required for specific operations and just write code blindly. This can cause things like the famous nested-loops effect instead of efficient joins, but this is solvable trough knowing how to properly use your tool. That being said ORMs are good at poking at small amounts of data (single or few objects), while SQL is probably the best language there is for huge set-based operations. When you are doing set-based things, not using SQL will most likely be a bad choice.
tom_b 17y ago
Good to know I'm not the only SQL junkie out there. So far, I'm making a pretty decent living eliminating thousands of lines of code by leveraging all that SQL gives me for free. I continue to be surprised by the number of hackers who don't know SQL well. And when developers say you can only really benefit with RDBMS when you have petabytes of data, I wonder if they haven't worked on apps that are explicitly data-centric. Granted, I spend my work days in apps that are or closely resemble data warehouses, but I can get your standard RDBMS engine to do a tremendous amount of work before I have to touch the first line of procedural code. If it fits your needs, by all means jump feet first into the RDBMS of your choice. When it doesn't fit, know enough to realize it and make a considered decision (seldo, +1 for mentioning the huge number of apps that don't need RDBMS) The point I'm really trying to express is that any hacker that touches a database would probably be well served to spend some minimal amount of effort learning ANSI SQL and understanding basic indexing, basic normal forms, and maybe star schemas. Adding some intermediate SQL and RDBMS skills to your toolbox is just another step towards maximizing your effectiveness as a developer.

by seldo 17y ago

dagheti 17y ago

I think part of the backlash is coming from developers on web-applications that have business models that require scaling to very large loads before they make sense. This puts a premium on keeping things very cheap (no heavy commercial SQLDBMS), and inherently scalable (hierarchical). Web developers on these types of projects cannot either a: buy a "real" SQLDBMS, or b: charge their customers so their business model makes sense without a huge number of transactions. This means they are willing to suffer it out with solutions that don't provide flexibility or data correctness guarantees of a RDBMS. This is just a guess, but it's quite likely that it is the rise of hyper-scale-intensive, speed-intensive, revenue-light business models that drives devs towards deciding SQL isn't worth it.

silentbicycle 17y ago
Postgresql is free. I'm not going to claim it's perfect, but it's pretty damn good, and postgres + intelligently used caching (such as memcached) goes a long way.
- kragen 17y ago
  Well, you're right that licensing cost can't be the reason, although ten years ago it could have been. But I think the "inherently scalable" item may have something to it.

TallGuyShort 17y ago

You're absolutely right. In my project, we were using a Java API to get some numbers from a huge database, but the calls took up way to many resources (time and memory). As an experiment, I tried querying the database directly in my app (bypassing the abstracted framework) and it was lightning fast. My code was hard to understand (and very hard to write, at times) but the result was a much lighter, much faster application.

wlievens 17y ago
So you need a framework that allows you to do both, without it falling apart.

xenophanes 17y ago

> 1. modern programming frameworks all abstracted away SQL because people found it hard to understand No. Not because it's complicated. Because they wanted higher level facilities for getting/setting data. In other words, they put in abstraction because abstraction is good, not because they are dumb.

dave_au 17y ago
There's probably two sets of people at work in the statement. The people who made the frameworks would require a good understand as well as the idea that an abstraction might make life easier. The side effect is that if you create a good enough abstraction you'll get people who can handle the abstraction but are a little shakey when dealing with what is abstracted. See, you can both be right :)
seldo 17y ago
Abstraction is good, but I think the problem is that ORM is not a good abstraction for anything but the trivial case of a table. As soon as you want to do anything actually relational, I have yet to see an ORM layer that comes up with an efficient join. The reason, say, C++ is a good abstraction for machine code is because it simplifies what's going on to the point that you can write something very complex in fewer lines of code, allowing it to fit into the human brain better. I'm willing to be shown examples to the contrary, but ORM layers don't simplify or compress relational queries, they just change the syntax to something more "object-y". So maybe my point is that abstraction is good, but ORM isn't really abstraction.
- trezor 17y ago
  ORM is not a good abstraction for anything but the trivial case of a table. As soon as you want to do anything actually relational, I have yet to see an ORM layer that comes up with an efficient join. Microsoft's Linq for SQL generates mostly OK SQL from your code. While not perfect for all complex operations (like multiple joins with aggregations), the SQL isn't horrible. For most use cases though, the SQL seems to be on par with what I would write myself. Ofcourse this is a rather vendor-specific implementation (for Microsoft SQL Server), but there are decent ORMs out there. The biggest problem really comes when people don't understand what underlying SQL will be required for specific operations and just write code blindly. This can cause things like the famous nested-loops effect instead of efficient joins, but this is solvable trough knowing how to properly use your tool. That being said ORMs are good at poking at small amounts of data (single or few objects), while SQL is probably the best language there is for huge set-based operations. When you are doing set-based things, not using SQL will most likely be a bad choice.

tom_b 17y ago

Good to know I'm not the only SQL junkie out there. So far, I'm making a pretty decent living eliminating thousands of lines of code by leveraging all that SQL gives me for free. I continue to be surprised by the number of hackers who don't know SQL well. And when developers say you can only really benefit with RDBMS when you have petabytes of data, I wonder if they haven't worked on apps that are explicitly data-centric. Granted, I spend my work days in apps that are or closely resemble data warehouses, but I can get your standard RDBMS engine to do a tremendous amount of work before I have to touch the first line of procedural code. If it fits your needs, by all means jump feet first into the RDBMS of your choice. When it doesn't fit, know enough to realize it and make a considered decision (seldo, +1 for mentioning the huge number of apps that don't need RDBMS) The point I'm really trying to express is that any hacker that touches a database would probably be well served to spend some minimal amount of effort learning ANSI SQL and understanding basic indexing, basic normal forms, and maybe star schemas. Adding some intermediate SQL and RDBMS skills to your toolbox is just another step towards maximizing your effectiveness as a developer.