4 ms·

Yeah, what a disappointment. In the early days of NoSql, a lot of dev were still shoehorning graphs, objects, or giant look up tables into SQL database, and ye

by geebee 9y ago

Yeah, what a disappointment. In the early days of NoSql, a lot of dev were still shoehorning graphs, objects, or giant look up tables into SQL database, and yes, it turns out that there are some wonderful databases to store and query this sort of data that don't rely on a relational model. And then... yep, a bunch of magpies decided that SQL shouldn't ever be used. It's important to point out that plenty of people who pioneered and promoted graph databases, object databases, and so forth didn't go down this rabbit hole. When the data was relational, plenty of these folks were all for relational databases. But I don't think you can claim the "NoSQL => never use SQL => SQL is obsolete" crowed was just a few outliers. This opinion infected a lot of software development organizations, and did plenty of harm. I love SQL, but let's not get too enthusiastic about sending the pendulum swinging back the other direction, even if tempting to watch it knock a few of the overly zealous NoSql types off their perches. Trust me, I still feel a flash of anger inside when I think of some of the arguments I had to go through when I advocated for relational SQL databases. But I don't want to be that guy myself - plenty of non-sql data bases are really well conceived and designed, and are ideal for the problems they address.

bunderbunder 9y ago
I would lay some of this at the feet of the fad for object-relational mapping, too. Trying to deal with object/relational impedance by sticking an object-oriented API in front of the database, and then using that to shoehorn OO ways of organizing and accessing data into an RDBMS, is a great way to place an upper bound on what kind of performance you can get out of the database. It also makes (what should be) routine schema migrations much more painful than they need to be. I think that that can explain a lot of the perceived success people were seeing with many document stores.
- geebee 9y ago
  That's a really good point. I have, certainly in Rails projects[1], seen the strangest databases. They are SQL databases that are useless outside the rails application. You actually can't query them in a relational sense, you have to go through the app and its methods of querying, transforming, and merging data, before you understand what it means. In short, the developers more or less used the ORM to serialize and object and write it out to disk. The persistent data basically makes no sense on its own, it has to repopulate the objects before it can be accessed in any meaningful way. There's little doubt that the devs behind these project had very little understanding of what an RDMBS actually is. And if you don't, then I suppose NoSql makes sense, because your data is never relational - not because it the relational model wouldn't be helpful, but because the developer can't see the relational model in the data stored, and is incapable of structuring it this way. I remember this idea (paraphrasing)[2] - the data will outlast the application, and the application will most likely outlast the developer. To me, this led to a design principle - the database should always be useful outside the context of the application. If you want information about the data you're storing, the DB should be a very, very useful place to get it. I'm not ruling out the value of the app! There are lots of ops you'd want to do as a combination of code and data, lots of UI, plenty of things. But if your data is useless and nonsensical unless your app transforms it, that is probably a sign of a big mistake[3] [1] I love rails, and I like ORMs. They save me so much typing compared to the old days of JDBC. It turns out it's not all that hard to create a well organized relational database through generators, and if you prefer to design the DB separately, it's not hard to map models to db tables. It really only gets out of control when people are essentially designing an RDBMS through generators and an ORM, but have no idea what idea what it all means on the backend. [2]https://blog.jooq.org/2014/01/02/why-your-data-will-outlast-your-sexy-new-technology/ https://blog.jooq.org/2014/01/02/why-your-data-will-outlast-... [3] probably
  - bdcravens 9y ago
    > That's a really good point. I have, certainly in Rails projects, seen the strangest databases. They are SQL databases that are useless outside the rails application. You actually can't query them in a relational sense, you have to go through the app and its methods of querying, transforming, and merging data, before you understand what it means. When I converted our application to Rails, I had a database that was pretty home-grown, as we were using a language/framework where you tended to write raw SQL. While I was able to tweak the model files to work with our schema, in most places I found it too challenging to construct ActiveRecord queries, and ended up writing a lot of raw SQL (or rather, copying it from previous iteration). Initially I thought I was doing it wrong because it wasn't Railsie enough, but realized that our database is our asset, and Rails is secondary. We do a lot of ad-hoc querying and report writing, so I definitely think database-first is the way to go in our use case.
- bdcravens 9y ago
  Indeed. I'm redesigning a database in Postgresql, and I'm taking the approach of building the best schema, and I'll tweak the ORM (ActiveRecord in Rails in our case)to work with that, as opposed to the other way around. (Currently we're probably 80% raw SQL)
- goalieca 9y ago
  I’m not sure why object oriented databases were a thing other than people liked to write OOP. The fact there is such a huge impedance between the structured data and management code seems to be a huge demonstration of cognitive dissonance in favour of OOP.
  - tabtab 9y ago
    Part of it is that DBA's got a bad reputation for bottlenecking projects for many different reasons. Some of it legitimate, as they wanted to make sure schema design was consistent, properly normalized, and followed org standards; but some DBA's were control-hungry. Therefore, developers tried to minimize their need to go through a DBA by doing more data-ish things inside the application itself. That's more of a staff management problem than a technology problem, but OOP had the "fad cred" of the time and often bowled over DBA's. The pendulum has started to swing the other way as reinventing the database & querying in applications has proved messy.
d0100 9y ago
Psql jsonb types is NoSQL enought for me. I now use it to great effect, instead of an EAV. On a side note, I was pretty excited that graphcool got open sourced, but their json limitation to 64kb is a dealbreaker.
bdcravens 9y ago
Increasingly I'm seeing NoSQL referred to as NOSQL - not "No SQL" but "Not Only SQL"
- geebee 9y ago
  I discussed this on HN a long time ago with one of the original creators/proponents of NoSql, and I believe this is what was intended. Sounds like a term that really got away from them.
  - bdcravens 9y ago
    > Sounds like a term that really got away from them. I think there's a class of developer that is very against "old" ways of doing things and has driven a lot of that narrative.
tabtab 9y ago
RE: "I love SQL": SQL as a language sucks in many ways, but so far there is no relational language that is ready to give it serious competition. The NoSQL movement was not really about the language, but about niches where processing or location scaling was more important than standard or even good query languages. The query language and hardware scaling are mostly two separate issues, and the NoSql movement unfortunately confused the two. It's mostly that SQL-based databases were slower to scale up, not that SQL is a horrible query language. As far as query languages, I've been in long discusses and debates about alternatives to SQL. I was partial to a draft language called "SMEQL", but everyone has different opinions on what a query language should look like and emphasize: it's hard to make everyone happy.
- flukus 9y ago
  Not really a fan of SMEQL, but I do wonder why we don't have more specialist languages that can transpile to sql. It might be the case that that a better sql only offers minor improvements and work needs to be done elsewhere, like in the hydration back into non-2d data, the current solutions I've seen require multiple repetitive sql statements and we might be better off making the hydrators smarter.

by geebee 9y ago

bunderbunder 9y ago

I would lay some of this at the feet of the fad for object-relational mapping, too. Trying to deal with object/relational impedance by sticking an object-oriented API in front of the database, and then using that to shoehorn OO ways of organizing and accessing data into an RDBMS, is a great way to place an upper bound on what kind of performance you can get out of the database. It also makes (what should be) routine schema migrations much more painful than they need to be. I think that that can explain a lot of the perceived success people were seeing with many document stores.

geebee 9y ago
That's a really good point. I have, certainly in Rails projects[1], seen the strangest databases. They are SQL databases that are useless outside the rails application. You actually can't query them in a relational sense, you have to go through the app and its methods of querying, transforming, and merging data, before you understand what it means. In short, the developers more or less used the ORM to serialize and object and write it out to disk. The persistent data basically makes no sense on its own, it has to repopulate the objects before it can be accessed in any meaningful way. There's little doubt that the devs behind these project had very little understanding of what an RDMBS actually is. And if you don't, then I suppose NoSql makes sense, because your data is never relational - not because it the relational model wouldn't be helpful, but because the developer can't see the relational model in the data stored, and is incapable of structuring it this way. I remember this idea (paraphrasing)[2] - the data will outlast the application, and the application will most likely outlast the developer. To me, this led to a design principle - the database should always be useful outside the context of the application. If you want information about the data you're storing, the DB should be a very, very useful place to get it. I'm not ruling out the value of the app! There are lots of ops you'd want to do as a combination of code and data, lots of UI, plenty of things. But if your data is useless and nonsensical unless your app transforms it, that is probably a sign of a big mistake[3] [1] I love rails, and I like ORMs. They save me so much typing compared to the old days of JDBC. It turns out it's not all that hard to create a well organized relational database through generators, and if you prefer to design the DB separately, it's not hard to map models to db tables. It really only gets out of control when people are essentially designing an RDBMS through generators and an ORM, but have no idea what idea what it all means on the backend. [2]https://blog.jooq.org/2014/01/02/why-your-data-will-outlast-your-sexy-new-technology/ https://blog.jooq.org/2014/01/02/why-your-data-will-outlast-... [3] probably
- bdcravens 9y ago
  > That's a really good point. I have, certainly in Rails projects, seen the strangest databases. They are SQL databases that are useless outside the rails application. You actually can't query them in a relational sense, you have to go through the app and its methods of querying, transforming, and merging data, before you understand what it means. When I converted our application to Rails, I had a database that was pretty home-grown, as we were using a language/framework where you tended to write raw SQL. While I was able to tweak the model files to work with our schema, in most places I found it too challenging to construct ActiveRecord queries, and ended up writing a lot of raw SQL (or rather, copying it from previous iteration). Initially I thought I was doing it wrong because it wasn't Railsie enough, but realized that our database is our asset, and Rails is secondary. We do a lot of ad-hoc querying and report writing, so I definitely think database-first is the way to go in our use case.
bdcravens 9y ago
Indeed. I'm redesigning a database in Postgresql, and I'm taking the approach of building the best schema, and I'll tweak the ORM (ActiveRecord in Rails in our case)to work with that, as opposed to the other way around. (Currently we're probably 80% raw SQL)
goalieca 9y ago
I’m not sure why object oriented databases were a thing other than people liked to write OOP. The fact there is such a huge impedance between the structured data and management code seems to be a huge demonstration of cognitive dissonance in favour of OOP.
- tabtab 9y ago
  Part of it is that DBA's got a bad reputation for bottlenecking projects for many different reasons. Some of it legitimate, as they wanted to make sure schema design was consistent, properly normalized, and followed org standards; but some DBA's were control-hungry. Therefore, developers tried to minimize their need to go through a DBA by doing more data-ish things inside the application itself. That's more of a staff management problem than a technology problem, but OOP had the "fad cred" of the time and often bowled over DBA's. The pendulum has started to swing the other way as reinventing the database & querying in applications has proved messy.

d0100 9y ago

Psql jsonb types is NoSQL enought for me. I now use it to great effect, instead of an EAV. On a side note, I was pretty excited that graphcool got open sourced, but their json limitation to 64kb is a dealbreaker.

bdcravens 9y ago

Increasingly I'm seeing NoSQL referred to as NOSQL - not "No SQL" but "Not Only SQL"

geebee 9y ago
I discussed this on HN a long time ago with one of the original creators/proponents of NoSql, and I believe this is what was intended. Sounds like a term that really got away from them.
- bdcravens 9y ago
  > Sounds like a term that really got away from them. I think there's a class of developer that is very against "old" ways of doing things and has driven a lot of that narrative.

tabtab 9y ago

RE: "I love SQL": SQL as a language sucks in many ways, but so far there is no relational language that is ready to give it serious competition. The NoSQL movement was not really about the language, but about niches where processing or location scaling was more important than standard or even good query languages. The query language and hardware scaling are mostly two separate issues, and the NoSql movement unfortunately confused the two. It's mostly that SQL-based databases were slower to scale up, not that SQL is a horrible query language. As far as query languages, I've been in long discusses and debates about alternatives to SQL. I was partial to a draft language called "SMEQL", but everyone has different opinions on what a query language should look like and emphasize: it's hard to make everyone happy.

flukus 9y ago
Not really a fan of SMEQL, but I do wonder why we don't have more specialist languages that can transpile to sql. It might be the case that that a better sql only offers minor improvements and work needs to be done elsewhere, like in the hydration back into non-2d data, the current solutions I've seen require multiple repetitive sql statements and we might be better off making the hydrators smarter.