7 ms·

> Let's start with the simplest one. Making the developer interface to the database a structured format instead of a textual query language was a clear win. I

by s_kilk 11y ago

> Let's start with the simplest one. Making the developer interface to the database a structured format instead of a textual query language was a clear win. I think this is the most significant factor, by far. With Mongo it's turtles (or at least Maps/Hashes) all the way down, without a strange pseudo-english layer near the bottom that forces you to translate back and forth. For some devs that's a big deal. For the last while I've been experimenting with bringing the same feature to PostgreSQL (http://bedquiltdb.github.io http://bedquiltdb.github.io), turns out it's very do-able, but I don't have enough time to make it as featureful as it needs to be.

collyw 11y ago
SQL is still one of the most readable languages in my opinion. Its the one language where I find it easier to read queries than write them.
- s_kilk 11y ago
  I should clarify, I'm a big fan of SQL and relational DBs, but it's hard to deny that some/many devs find the abstraction boundary a bit weird. And really, when you think about it, it is a bit weird, (we send sentences of almost-english toward the DB, then get structured data back) and there's nothing wrong with that. But many devs do value the ability to stay in hash-map land all day without needing to think about how to cross an abstraction boundary into the DB, and MongoDB actually has a pretty cool solution there.
- pmelendez 11y ago
  SQL is fine, no problem with it unless you are embedding queries in a OO application and you are forced to hack mapping layer between the two interfaces, which gives you two options, either compromise performance for the sake of development easiness or vice versa.
  - halostatue 11y ago
    I’ve played a bit with Jeremy Evans’s Sequel (http://sequel.jeremyevans.net http://sequel.jeremyevans.net), and I’m not sure that ORMs are necessarily constrained along this dichotomy (easy to use but with poor performance vs great performance but hard to use). Sequel seems to be about making the hard things easy and reducing the amount of time you need to drop to pure SQL to almost zero. As someone who has built with Oracle’s Pro C(++) and with various ORMs, Sequel is an ORM that makes me reasonably happy and gives me the expressiveness of embedded SQL without compromising my object model.
  - arohner 11y ago
    > SQL is fine, no problem with it unless you are embedding queries in a OO application which is essentially every single production deployment of a SQL database in the last twenty years. That's the problem.
    - collyw 11y ago
      A lot of people are using ORM's.
- jerf 11y ago
  SQL's fine on its own. The problem arises when you try to manipulate it. A query to get messages that were sent between now and a week ago, joining in the user table for both the sender and the receiver to get their full names, is sensible enough when written out as SQL. I can quibble about some of the affordances of SQL, but it's good enough for most queries. The problem is if you try to create some sort of data that represents "get the message body", "messages sent since last week", "get the sender's name", "get the receiver's name", and somehow programmatically assemble a query out of the pieces. (Slight quibble, this isn't actually about OO, this is about "anything that isn't SQL". A Haskell library will have the exact same problem.) An AST-based approach doesn't guarantee that this will be easy, but it's still a step above trying to assemble an SQL query from those pieces. It's totally possible. There's a ton of libraries that do it in various ways. It's just not even remotely easy, like it should be. If you look into the inside of those libraries you'll find a C'thulian monstronsity of special cases interacting with special cases until the whole thing just explodes into a brain-consuming mess, because SQL was very, very clearly not written for this use case. In another 10 or 20 years I look forward to data-based analysis that tries to determine how much of the "NoSQL" movement was because the relational data model doesn't work for all use cases, and how much of it was the entirely accidental (in the Brooks sense) problems with A: SQL, the language itself, not its capabilities and B: schema migrations with no essential reason to be as painful as they are. (And something something column stores, but I'm not sure where they fit into this story exactly.) And to be clear on tone, I really am interested. I'm pretty sure the answer won't be either extreme but I'm pretty uncertain about where in the middle we'll fall.
  - virtualwhys 11y ago
    > and somehow programmatically assemble a query out of the pieces. You can do this today, in various statically typed languages[1] (perhaps dynamic languages as well, though the composition will likely be more ad hoc). Agreed re: under the hood complexity, but that's more in supporting multiple database engines (and related corner cases) than the transformation of Query > AST > SQL. [1] https://news.ycombinator.com/item?id=10525040 https://news.ycombinator.com/item?id=10525040
    - 11y ago
anton_gogolev 11y ago
> Making the developer interface to the database a structured format instead of a textual query language was a clear win. No, it was not. It is an abomiation, just like SharePoint CAML Query or any other "express-your-query-as-an-AST" approach. Sure, it's a paradise for Buzzword-Compliant Fluent Interface Enterprise Query Builder-type libraries. > Querying SQL data involves constructing strings... No. Case in point: LINQ. How it is implemented is irrelevant to the statement above. > ...and counting arguments very carefully No. Named parameters to the rescue.
- dkhenry 11y ago
  LINQ is just an "express your query as an AST" approach. Sure LINQ presents its self as a DSL on top of that AST ( which is exactly what SQL is at the end of the day ), but there are libraries for MongoDB that allow you to do the same thing. There is even a LINQ driver for MongoDB[1]. What you are missing is that by having the lowest common denominator be the AST instead of a DSL you have removed a huge burden from both the server and client side of the operation. From that AST you can build any kind of client side interface without worry about how to parse or compile the result into a specific textual language, even plain old SQL[2] can be used if you like your sever spending its clock cycles parsing text. 1. http://mongodb.github.io/mongo-csharp-driver/2.1/reference/driver/crud/linq/ http://mongodb.github.io/mongo-csharp-driver/2.1/reference/d... 2. https://www.progress.com/connectors/mongodb https://www.progress.com/connectors/mongodb
  - tracker1 11y ago
    I like mongo's interface (most of the time)... but when you get to more interesting queries, it gets far more weird than even SQL... I wrote an early node API almost mirroring the JSON query interface (was easy enough to do filtering on sensitive fields), and it worked really well. That said, it was really clunky dealing with ordering as JSON doesn't guarantee serialization order (sort fields), and that was cumbersome to work around iirc... Today RethinkDB would be my first choice for a similar solution... better ops interfaces with replication + failover. Instead of having to do a replica of a sharded system in mongo for distribution with redundancy. Also, while the query interface is a little more complicated to get started with, when your queries get more interesting, it's not nearly as messy as mongo gets.
dbattaglia 11y ago
Maybe I'm weird and just like SQL but the query-as-data aspect of Mongo is actually my least favorite aspect of using it. Ad-hoc queries become torture, as do complicated queries. I might just be lucky to have used SQL Server and C# for so long which eases a lot of the pains you described.
- meowface 11y ago
  Same. I wish there was a DSL layer on top.
  - leothekim 11y ago
    If you use Scala and mongodb, I highly recommend this DSL: https://github.com/foursquare/rogue https://github.com/foursquare/rogue Typesafe queries using SQL-like syntax. Particularly when used with the Spindle library (https://github.com/foursquare/spindle https://github.com/foursquare/spindle), this is best ORM I've ever used.
    - meowface 11y ago
      That does look very nice. I've never actually used Scala seriously, but coincidentally have been learning how to use it all this week. I'm even more interested, now.
    - meowface 11y ago
      That does look very nice. I've never actually used Scala seriously, but coincidentally have been learning how to use it all this week. I'm even more interested, now.
- lloyd-christmas 11y ago
  It's pretty easy to build wrappers for things like that, though. Build a constructor that has instances methods of just doing `myQueryInstance.addEqualsQuery(key, value)`. You can then just pass it around and add to the query instance as it moves around, deconstructing it at your database access point.
yoklov 11y ago
Not a fan of mongo's query format/language, but I have to agree. It's always bugged me that so much work has gone into relational databases, and yet the only way anybody interfaces with them is by using an extremely quirky programming language from the 70's (or whenever).
- gedrap 11y ago
  >>> extremely quirky programming language from the 70's (or whenever) Because it doesn't have to be shinny and all, it just works. There are no widely used alternatives, so we can say that people are happy-enough using it. See the situation with JS. CoffeeScript and all, it's quite popular. Is there an analogy (in terms of functionality and popularity) in SQL? Nope. No one bothered to do it well enough, or not enough people found that it's worth enough to learn it. We don't need to reinvent something every few years just because new JS frameworks come out and others are forgotten every few months.
- collyw 11y ago
  Well you do have MS Access query builder, which I used for a while. Once you got used to it, it was maybe a bit faster than using real SQL. But then it locked you into the query builder way of thinking, and didn't provide very much beneft over textual SQL.
gedrap 11y ago
If you've been programming and using SQL for a few years, it's not a problem. It might take a bit of time to get the feeling of SQL, but it's ok later. And that's fine. rails-15min-blog things are cool and all, but some things have a learning curve and that's ok. The vast vast majority of SQL in the wild is pretty straightforward and sucks only when the schema sucks (can't blame language for that). Most of the time people hate SQL, they do so because they are working with crap schema (whether someone else created it or they themselves didn't think it through enough). Sane schema + putting a bit of effort write readable SQL can go a long long way.
- snaily 11y ago
  Given how quickly software engineering as a discipline moves, it's not unreasonable that most users at any given point in time (for any given library) are in those first few years. Optimizing for the "newbie" case is not a failure.
  - gedrap 11y ago
    Absolutely. But optimizing for the long term (and it doesn't seem like SQL is going to disappear any time soon) is not a failure either :) If a newbie asked me whether to learn MySQL/PostgreSQL, or MongoDB, I'd absolutely say *SQL. Yeah, it's not as straightforward as MongoDB. It's not intuitive for many. But this knowledge will still be useful a year, or 5 years later. Finding a job will be easier too. So, yeah, it could be better, but it's still a good, safe choice.
    - jrumbut 11y ago
      Am I alone in finding SQL/normalized relational databases to be easier to learn? In the relational world, you can get a lot of insight into what will and won't work with simple math. There certainly are surprises in every database, but most things are not surprises. NoSQL databases tend not to leverage such a clear and complete model. You have to study each implementation to learn where the surprises are, and there are more of them. That's not to say they don't both have their role, I just wonder if people who have a little (and in my care very little) CS theory still remembered find RDBMSes easier than those who have learned development without theory.
xlm1717 11y ago
For this dev, it's a big deal. I love writing a query in MongoDB way more than I love writing a query in MySQL. Granted, I think a big part of it has to do with joins being the annoying part of SQL...
- collyw 11y ago
  That seems to add to the idea that NoSQL fans just don't know how to use SQL properly.
  - obsurveyor 11y ago
    Are you implying you're not supposed to use joins in SQL?
    - astine 11y ago
      I think he's implying that finding joins difficult is a sign of immaturity of the developer rather than a problem with the language. Joins might be tricky if you are unused to them, but they are the secret sauce of relational databases. They are what make it possible to have dozens of views on a single normalized data structure.
  - woah 11y ago
    This just adds to the idea that sql kind of sucks in terms of usability. For the record I always use sql, but let's be honest with ourselves.
- acjohnson55 11y ago
  What's annoying about joins? I think practically speaking, whether you're going relational or not, joins are a reality of a data model that supports multiple views and transformations. Assuming that's the case, the only question is whether you're doing joins ad hoc in your application code or you've learned the handful of variations that work natively in your RDBMS.
ThePhysicist 11y ago
I actually built a MongoDB query interface for SQL via Python+SQLAlchemy. It allows you to query your relational database just like you would with MongoDB, and in addition use some things (e.g. Joins in queries or "deep querying") that are not possible using MongoDB. Here's the repo: https://github.com/adewes/blitzdb https://github.com/adewes/blitzdb The implementation is stable but I'm still working on finishing Python3 support and documentation.

by s_kilk 11y ago

collyw 11y ago

SQL is still one of the most readable languages in my opinion. Its the one language where I find it easier to read queries than write them.

s_kilk 11y ago
I should clarify, I'm a big fan of SQL and relational DBs, but it's hard to deny that some/many devs find the abstraction boundary a bit weird. And really, when you think about it, it is a bit weird, (we send sentences of almost-english toward the DB, then get structured data back) and there's nothing wrong with that. But many devs do value the ability to stay in hash-map land all day without needing to think about how to cross an abstraction boundary into the DB, and MongoDB actually has a pretty cool solution there.
pmelendez 11y ago
SQL is fine, no problem with it unless you are embedding queries in a OO application and you are forced to hack mapping layer between the two interfaces, which gives you two options, either compromise performance for the sake of development easiness or vice versa.
- halostatue 11y ago
  I’ve played a bit with Jeremy Evans’s Sequel (http://sequel.jeremyevans.net http://sequel.jeremyevans.net), and I’m not sure that ORMs are necessarily constrained along this dichotomy (easy to use but with poor performance vs great performance but hard to use). Sequel seems to be about making the hard things easy and reducing the amount of time you need to drop to pure SQL to almost zero. As someone who has built with Oracle’s Pro C(++) and with various ORMs, Sequel is an ORM that makes me reasonably happy and gives me the expressiveness of embedded SQL without compromising my object model.
- arohner 11y ago
  > SQL is fine, no problem with it unless you are embedding queries in a OO application which is essentially every single production deployment of a SQL database in the last twenty years. That's the problem.
  - collyw 11y ago
    A lot of people are using ORM's.
jerf 11y ago
SQL's fine on its own. The problem arises when you try to manipulate it. A query to get messages that were sent between now and a week ago, joining in the user table for both the sender and the receiver to get their full names, is sensible enough when written out as SQL. I can quibble about some of the affordances of SQL, but it's good enough for most queries. The problem is if you try to create some sort of data that represents "get the message body", "messages sent since last week", "get the sender's name", "get the receiver's name", and somehow programmatically assemble a query out of the pieces. (Slight quibble, this isn't actually about OO, this is about "anything that isn't SQL". A Haskell library will have the exact same problem.) An AST-based approach doesn't guarantee that this will be easy, but it's still a step above trying to assemble an SQL query from those pieces. It's totally possible. There's a ton of libraries that do it in various ways. It's just not even remotely easy, like it should be. If you look into the inside of those libraries you'll find a C'thulian monstronsity of special cases interacting with special cases until the whole thing just explodes into a brain-consuming mess, because SQL was very, very clearly not written for this use case. In another 10 or 20 years I look forward to data-based analysis that tries to determine how much of the "NoSQL" movement was because the relational data model doesn't work for all use cases, and how much of it was the entirely accidental (in the Brooks sense) problems with A: SQL, the language itself, not its capabilities and B: schema migrations with no essential reason to be as painful as they are. (And something something column stores, but I'm not sure where they fit into this story exactly.) And to be clear on tone, I really am interested. I'm pretty sure the answer won't be either extreme but I'm pretty uncertain about where in the middle we'll fall.
- virtualwhys 11y ago
  > and somehow programmatically assemble a query out of the pieces. You can do this today, in various statically typed languages[1] (perhaps dynamic languages as well, though the composition will likely be more ad hoc). Agreed re: under the hood complexity, but that's more in supporting multiple database engines (and related corner cases) than the transformation of Query > AST > SQL. [1] https://news.ycombinator.com/item?id=10525040 https://news.ycombinator.com/item?id=10525040
  - 11y ago

anton_gogolev 11y ago

> Making the developer interface to the database a structured format instead of a textual query language was a clear win. No, it was not. It is an abomiation, just like SharePoint CAML Query or any other "express-your-query-as-an-AST" approach. Sure, it's a paradise for Buzzword-Compliant Fluent Interface Enterprise Query Builder-type libraries. > Querying SQL data involves constructing strings... No. Case in point: LINQ. How it is implemented is irrelevant to the statement above. > ...and counting arguments very carefully No. Named parameters to the rescue.

dkhenry 11y ago
LINQ is just an "express your query as an AST" approach. Sure LINQ presents its self as a DSL on top of that AST ( which is exactly what SQL is at the end of the day ), but there are libraries for MongoDB that allow you to do the same thing. There is even a LINQ driver for MongoDB[1]. What you are missing is that by having the lowest common denominator be the AST instead of a DSL you have removed a huge burden from both the server and client side of the operation. From that AST you can build any kind of client side interface without worry about how to parse or compile the result into a specific textual language, even plain old SQL[2] can be used if you like your sever spending its clock cycles parsing text. 1. http://mongodb.github.io/mongo-csharp-driver/2.1/reference/driver/crud/linq/ http://mongodb.github.io/mongo-csharp-driver/2.1/reference/d... 2. https://www.progress.com/connectors/mongodb https://www.progress.com/connectors/mongodb
- tracker1 11y ago
  I like mongo's interface (most of the time)... but when you get to more interesting queries, it gets far more weird than even SQL... I wrote an early node API almost mirroring the JSON query interface (was easy enough to do filtering on sensitive fields), and it worked really well. That said, it was really clunky dealing with ordering as JSON doesn't guarantee serialization order (sort fields), and that was cumbersome to work around iirc... Today RethinkDB would be my first choice for a similar solution... better ops interfaces with replication + failover. Instead of having to do a replica of a sharded system in mongo for distribution with redundancy. Also, while the query interface is a little more complicated to get started with, when your queries get more interesting, it's not nearly as messy as mongo gets.

dbattaglia 11y ago

Maybe I'm weird and just like SQL but the query-as-data aspect of Mongo is actually my least favorite aspect of using it. Ad-hoc queries become torture, as do complicated queries. I might just be lucky to have used SQL Server and C# for so long which eases a lot of the pains you described.

meowface 11y ago
Same. I wish there was a DSL layer on top.
- leothekim 11y ago
  If you use Scala and mongodb, I highly recommend this DSL: https://github.com/foursquare/rogue https://github.com/foursquare/rogue Typesafe queries using SQL-like syntax. Particularly when used with the Spindle library (https://github.com/foursquare/spindle https://github.com/foursquare/spindle), this is best ORM I've ever used.
  - meowface 11y ago
    That does look very nice. I've never actually used Scala seriously, but coincidentally have been learning how to use it all this week. I'm even more interested, now.
  - meowface 11y ago
    That does look very nice. I've never actually used Scala seriously, but coincidentally have been learning how to use it all this week. I'm even more interested, now.
lloyd-christmas 11y ago
It's pretty easy to build wrappers for things like that, though. Build a constructor that has instances methods of just doing `myQueryInstance.addEqualsQuery(key, value)`. You can then just pass it around and add to the query instance as it moves around, deconstructing it at your database access point.

yoklov 11y ago

Not a fan of mongo's query format/language, but I have to agree. It's always bugged me that so much work has gone into relational databases, and yet the only way anybody interfaces with them is by using an extremely quirky programming language from the 70's (or whenever).

gedrap 11y ago
>>> extremely quirky programming language from the 70's (or whenever) Because it doesn't have to be shinny and all, it just works. There are no widely used alternatives, so we can say that people are happy-enough using it. See the situation with JS. CoffeeScript and all, it's quite popular. Is there an analogy (in terms of functionality and popularity) in SQL? Nope. No one bothered to do it well enough, or not enough people found that it's worth enough to learn it. We don't need to reinvent something every few years just because new JS frameworks come out and others are forgotten every few months.
collyw 11y ago
Well you do have MS Access query builder, which I used for a while. Once you got used to it, it was maybe a bit faster than using real SQL. But then it locked you into the query builder way of thinking, and didn't provide very much beneft over textual SQL.

gedrap 11y ago

If you've been programming and using SQL for a few years, it's not a problem. It might take a bit of time to get the feeling of SQL, but it's ok later. And that's fine. rails-15min-blog things are cool and all, but some things have a learning curve and that's ok. The vast vast majority of SQL in the wild is pretty straightforward and sucks only when the schema sucks (can't blame language for that). Most of the time people hate SQL, they do so because they are working with crap schema (whether someone else created it or they themselves didn't think it through enough). Sane schema + putting a bit of effort write readable SQL can go a long long way.

snaily 11y ago
Given how quickly software engineering as a discipline moves, it's not unreasonable that most users at any given point in time (for any given library) are in those first few years. Optimizing for the "newbie" case is not a failure.
- gedrap 11y ago
  Absolutely. But optimizing for the long term (and it doesn't seem like SQL is going to disappear any time soon) is not a failure either :) If a newbie asked me whether to learn MySQL/PostgreSQL, or MongoDB, I'd absolutely say *SQL. Yeah, it's not as straightforward as MongoDB. It's not intuitive for many. But this knowledge will still be useful a year, or 5 years later. Finding a job will be easier too. So, yeah, it could be better, but it's still a good, safe choice.
  - jrumbut 11y ago
    Am I alone in finding SQL/normalized relational databases to be easier to learn? In the relational world, you can get a lot of insight into what will and won't work with simple math. There certainly are surprises in every database, but most things are not surprises. NoSQL databases tend not to leverage such a clear and complete model. You have to study each implementation to learn where the surprises are, and there are more of them. That's not to say they don't both have their role, I just wonder if people who have a little (and in my care very little) CS theory still remembered find RDBMSes easier than those who have learned development without theory.

xlm1717 11y ago

For this dev, it's a big deal. I love writing a query in MongoDB way more than I love writing a query in MySQL. Granted, I think a big part of it has to do with joins being the annoying part of SQL...

collyw 11y ago
That seems to add to the idea that NoSQL fans just don't know how to use SQL properly.
- obsurveyor 11y ago
  Are you implying you're not supposed to use joins in SQL?
  - astine 11y ago
    I think he's implying that finding joins difficult is a sign of immaturity of the developer rather than a problem with the language. Joins might be tricky if you are unused to them, but they are the secret sauce of relational databases. They are what make it possible to have dozens of views on a single normalized data structure.
- woah 11y ago
  This just adds to the idea that sql kind of sucks in terms of usability. For the record I always use sql, but let's be honest with ourselves.
acjohnson55 11y ago
What's annoying about joins? I think practically speaking, whether you're going relational or not, joins are a reality of a data model that supports multiple views and transformations. Assuming that's the case, the only question is whether you're doing joins ad hoc in your application code or you've learned the handful of variations that work natively in your RDBMS.

ThePhysicist 11y ago

I actually built a MongoDB query interface for SQL via Python+SQLAlchemy. It allows you to query your relational database just like you would with MongoDB, and in addition use some things (e.g. Joins in queries or "deep querying") that are not possible using MongoDB. Here's the repo: https://github.com/adewes/blitzdb https://github.com/adewes/blitzdb The implementation is stable but I'm still working on finishing Python3 support and documentation.