6 ms·

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are

by taf2 13y ago

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

neumino 13y ago
ReQL is a new query language, and indeed requires the user to learn it but - It's actually easy to learn -- especially with the data explorer - It's nicer to use than SQL -- You can chain commands in the language you are using
teraflop 13y ago
One reason is that SQL only lets you work with flat relational tables. MongoDB and RethinkDB let you store complex JSON-like objects as values, and extending SQL to query and manipulate those kinds of structures is hard. The closest attempt I've seen is Google's BigQuery. Another reason is that SQL does its best to decouple the query (what you're asking for) from the execution plan (how it actually gets evaluated). But for operations like joins, the precise way the query is evaluated can have a huge effect on performance. So it makes sense to design your query language to explicitly expose those knobs to the programmer.
- spamizbad 13y ago
  I have no experience with RethinkDB, so maybe they did it right, but I have quite a bit with Mongo. MongoDB's querying is nowhere near as powerful as SQL. And understanding its limitations and pain points are essential when structuring your data. Otherwise you'll never really scale unless you've got data that's embarrassingly easy to query (eg: filtering on 1 or 2 indexed fields). Being able to write JS map-reduce queries is fine when you need to do stuff ad-hoc, but none of it can be used in production at scale. Then there's the aggregate framework which helps quite a bit but still doesn't offer the same level of performance I'd expect from Postgres, MSSQL, or Oracle. Things get really yucky when you have to start unpacking arrays - especially since there's a memory limit on $sort and $group. There are further issues related to skipping being slow on large collections because it must step through. And to cope with these limitations you need to invest a ton of time into coding around them. The freedom of the database being schemaless? Gone. You must carefully structure data and decide up-front how it needs to be queried - or cope with poor performance. And those complex queries, when written in ugly JSON, aren't the most human-readable things in the world. Certainly not any better than SQL. I'd rather just invest the time in learning SQL than partake in the mental gymnastics necessary squeeze high-performance non-trivial queries out of MongoDB. I understand MongoDB is a big hit with people who use it for low volume internal tools or people building MVPs. I can totally understand how its an awesome tool for those. But rusty old SQL starts looking better when your app scales and the business demands change.
  - orthecreedence 13y ago
    You're right, you haven't tried REQL =]. As a developer of a RethinkDB driver and a fairly active user of the database, and also a previous fan of Mongo, I can tell you that REQL kicks the shit out of MongoDB's querying. SQL really is well-geared towards relational, flat database systems. Mongo's querying is an obtuse command-based language that always felt like it was adding hacks on hacks to get the data you wanted. REQL is almost like having your data in-memory, and you're running programmatic expressions on it that seamlessly melt into your native language. The lack of a query optimizer almost makes it better because you have to think about your query plans and indexes instead of just firing it off at the server and crossing your fingers. You're not running commands, you're processing data. I don't have a lot of experience scaling Rethink, but I do know from reading the docs and architecture that it scales out better than most SQL servers will and certainly scale up better than Mongo. I've tried to scale both MySQL and Mongo. Both are difficult and painful. You can't conclude that because of Mongo's failures, SQL beats Rethink. Rethink is light years ahead of Mongo.
- zapov 13y ago
  This is not true. Both Oracle and Postgres are ORDBMS and you can work with objects/nested collections and all kind of non-relational constructs.
coffeemug 13y ago
> I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. Actually, it's a really good question. I'm one of the ReQL designers at Rethink, and I was the one pushing for no SQL compatibility. Here is some of my reasoning (we could talk about this for days, though): * Even SQL designers would tell you SQL isn't a very good programming language. It even looks like Cobol! Imagine if every language after Cobol decided to be backwards compatible -- what sort of world would we live in? * SQL is really bad for querying hierarchical data with lots of empty columns. The quality of experience you get from learning a new language designed for JSON, far outweighs the downsides of learning it. We're assuming people will be using ReQL fifteen years from now. * Designing a language that embeds into your programming language wasn't possible before -- but it is now. That means no more SQL injection, no more string manipulation, no more heavy ORMs. Well worth it in my opinion. * The chaining paradigm (largely pinoneered and proven by jQuery) is magical for getting people to intuitively understand how to write complex queries. No more StackOverflow questions of "How do I do X in SQL?" because there is now an intuitive consistency to the language. * SQL compatibility is very, very difficult. The standard is hard to implement, and has lots of grey area around the edges. You can go for full bug-by-bug compatibility, which would take decades. Or you can go for basic compatibility, which confuses people. They try to port their application, it works for a while, and then breaks in some grey area in production, which results in a terrible user experience. As an example, I'd ponder on why SQL has both the `where` keyword, and the `having` keyword. This alone wouldn't be enough of a reason to design a new language, of course, but this sort of thing permeates SQL. It's 2014. We can do better.
- shawn-butler 13y ago
  HAVING specifies a search condition for a group or an aggregate function.
  - coffeemug 13y ago
    Sure, but consider how this works in ReQL: r.table('foo').filter(...) r.table('foo').group('category').filter(...) A properly designed language shouldn't have two different keywords for something that does ostensibly the same thing in different contexts. That's a mark of bad language design.
biokoda 13y ago
there are some that do. The so called newSql databases like actordb
willvarfar 13y ago
I completely agree. Postgres, for example, has nice new JSON support, demonstrating that SQL can be grown in new modern ways.
coolsunglasses 13y ago
For an idea of what's possible when you put aside SQL: https://github.com/bitemyapp/Revise/ https://github.com/bitemyapp/Revise/
fidotron 13y ago
I'm totally the opposite, and blame you entirely for provoking what follows! SQL at this point is passed expiry. The simple reason is we now have plenty of decent scripting languages to run as interactive prompts, all of which are far better at interfacing with the rest of the system than SQL is. Exposing a sane, direct, API in those languages gets you much more than SQL, along with having removed multiple layers of confusion, including string generation, escaping, reparsing, then is it doing what you want, etc. I've been using LevelDB (via plyvel) a lot lately for data storage, and every time I end up having to use SQL (even indirectly via ORM) is painful by comparison because you can just feel the control being taken away from you, and somewhere you end up having to fire up the DB prompt for no good reason such as adding strange DB specific indexing flags to columns or even setting up authorization and DB creation, making it yet another thing to go wrong during deployment. Protocol buffers stored in LevelDB prove so easy to use by comparison with something like SQLite or PgSQL just at an API level. The resulting code is simpler, cleaner, and much easier to reason about. If I need to move it to another format the code, again, is amazingly small. As someone that cares deeply about my app's data structure seeing the acceptance of a world beyond SQL is one of the best developments in my career, and experience means I simply don't trust any SQL based abstraction to give you the controls to get it right.
- coldtea 13y ago
  >I'm totally the opposite, and blame you entirely for provoking what follows! SQL at this point is passed expiry. Actually it's the very opposite. SQL is a higher level abstraction for all this ad-hoc junk, based on actual mathematical principles (relational algebra), and it's also declarative, instead of imperative. Furthermore, all this NoSQL query systems now emerging are nothing new. They were tried in the 70s and early 80s, and people found out that they sucked. Before SQL what we had was, well, NoSQL. Wanting a flexible schema-less db or one that's denormalised for certain needs (like Google's or Facebook's) makes sense. Replacing SQL and RBDMS for the common tasks they are used (company logistics and accounting, etc) with NoSQL is a regression to the primitive past. I guess people not knowning (computing) history are doomed to repeat it.
baruch 13y ago
I much prefer ReQL over SQL. It is much nicer, flows naturally in my opinion and is combine-able. I'm using this combination method in my rethink-miner tool to help me work with data to try and figure it out and I can make generic queries that I can use on top of every other sub-query which is very very nice.
j_s 13y ago
For now it probably keeps a number of users away that they don't want to support, but it makes sense to at least have an automatic translation layer in the long run.
chipotle_coyote 13y ago
I'm not really sure how you could wedge SQL into a system that doesn't really have the concept of columns without it being, at the least, a distinctly different dialect. And, really, if you've worked with some of the "higher level" interfaces to SQL like SQLAlchemy or Laravel's QueryBuilder, ReQL doesn't seem that alien; they all tend to work by method chaining as well. (But they usually don't support everything SQL does, unlike ReQL's interfaces.) Also: unlike my (admittedly limited) exposure to other NoSQL systems, RethinkDB supports relations and joins -- it's actually a lot easier for an SQL fiend to move to Rethink than any of its competitors.

by taf2 13y ago

neumino 13y ago

ReQL is a new query language, and indeed requires the user to learn it but - It's actually easy to learn -- especially with the data explorer - It's nicer to use than SQL -- You can chain commands in the language you are using

teraflop 13y ago

One reason is that SQL only lets you work with flat relational tables. MongoDB and RethinkDB let you store complex JSON-like objects as values, and extending SQL to query and manipulate those kinds of structures is hard. The closest attempt I've seen is Google's BigQuery. Another reason is that SQL does its best to decouple the query (what you're asking for) from the execution plan (how it actually gets evaluated). But for operations like joins, the precise way the query is evaluated can have a huge effect on performance. So it makes sense to design your query language to explicitly expose those knobs to the programmer.

spamizbad 13y ago
I have no experience with RethinkDB, so maybe they did it right, but I have quite a bit with Mongo. MongoDB's querying is nowhere near as powerful as SQL. And understanding its limitations and pain points are essential when structuring your data. Otherwise you'll never really scale unless you've got data that's embarrassingly easy to query (eg: filtering on 1 or 2 indexed fields). Being able to write JS map-reduce queries is fine when you need to do stuff ad-hoc, but none of it can be used in production at scale. Then there's the aggregate framework which helps quite a bit but still doesn't offer the same level of performance I'd expect from Postgres, MSSQL, or Oracle. Things get really yucky when you have to start unpacking arrays - especially since there's a memory limit on $sort and $group. There are further issues related to skipping being slow on large collections because it must step through. And to cope with these limitations you need to invest a ton of time into coding around them. The freedom of the database being schemaless? Gone. You must carefully structure data and decide up-front how it needs to be queried - or cope with poor performance. And those complex queries, when written in ugly JSON, aren't the most human-readable things in the world. Certainly not any better than SQL. I'd rather just invest the time in learning SQL than partake in the mental gymnastics necessary squeeze high-performance non-trivial queries out of MongoDB. I understand MongoDB is a big hit with people who use it for low volume internal tools or people building MVPs. I can totally understand how its an awesome tool for those. But rusty old SQL starts looking better when your app scales and the business demands change.
- orthecreedence 13y ago
  You're right, you haven't tried REQL =]. As a developer of a RethinkDB driver and a fairly active user of the database, and also a previous fan of Mongo, I can tell you that REQL kicks the shit out of MongoDB's querying. SQL really is well-geared towards relational, flat database systems. Mongo's querying is an obtuse command-based language that always felt like it was adding hacks on hacks to get the data you wanted. REQL is almost like having your data in-memory, and you're running programmatic expressions on it that seamlessly melt into your native language. The lack of a query optimizer almost makes it better because you have to think about your query plans and indexes instead of just firing it off at the server and crossing your fingers. You're not running commands, you're processing data. I don't have a lot of experience scaling Rethink, but I do know from reading the docs and architecture that it scales out better than most SQL servers will and certainly scale up better than Mongo. I've tried to scale both MySQL and Mongo. Both are difficult and painful. You can't conclude that because of Mongo's failures, SQL beats Rethink. Rethink is light years ahead of Mongo.
zapov 13y ago
This is not true. Both Oracle and Postgres are ORDBMS and you can work with objects/nested collections and all kind of non-relational constructs.

coffeemug 13y ago

> I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. Actually, it's a really good question. I'm one of the ReQL designers at Rethink, and I was the one pushing for no SQL compatibility. Here is some of my reasoning (we could talk about this for days, though): * Even SQL designers would tell you SQL isn't a very good programming language. It even looks like Cobol! Imagine if every language after Cobol decided to be backwards compatible -- what sort of world would we live in? * SQL is really bad for querying hierarchical data with lots of empty columns. The quality of experience you get from learning a new language designed for JSON, far outweighs the downsides of learning it. We're assuming people will be using ReQL fifteen years from now. * Designing a language that embeds into your programming language wasn't possible before -- but it is now. That means no more SQL injection, no more string manipulation, no more heavy ORMs. Well worth it in my opinion. * The chaining paradigm (largely pinoneered and proven by jQuery) is magical for getting people to intuitively understand how to write complex queries. No more StackOverflow questions of "How do I do X in SQL?" because there is now an intuitive consistency to the language. * SQL compatibility is very, very difficult. The standard is hard to implement, and has lots of grey area around the edges. You can go for full bug-by-bug compatibility, which would take decades. Or you can go for basic compatibility, which confuses people. They try to port their application, it works for a while, and then breaks in some grey area in production, which results in a terrible user experience. As an example, I'd ponder on why SQL has both the `where` keyword, and the `having` keyword. This alone wouldn't be enough of a reason to design a new language, of course, but this sort of thing permeates SQL. It's 2014. We can do better.

shawn-butler 13y ago
HAVING specifies a search condition for a group or an aggregate function.
- coffeemug 13y ago
  Sure, but consider how this works in ReQL: r.table('foo').filter(...) r.table('foo').group('category').filter(...) A properly designed language shouldn't have two different keywords for something that does ostensibly the same thing in different contexts. That's a mark of bad language design.

biokoda 13y ago

there are some that do. The so called newSql databases like actordb

willvarfar 13y ago

I completely agree. Postgres, for example, has nice new JSON support, demonstrating that SQL can be grown in new modern ways.

coolsunglasses 13y ago

For an idea of what's possible when you put aside SQL: https://github.com/bitemyapp/Revise/ https://github.com/bitemyapp/Revise/

fidotron 13y ago

I'm totally the opposite, and blame you entirely for provoking what follows! SQL at this point is passed expiry. The simple reason is we now have plenty of decent scripting languages to run as interactive prompts, all of which are far better at interfacing with the rest of the system than SQL is. Exposing a sane, direct, API in those languages gets you much more than SQL, along with having removed multiple layers of confusion, including string generation, escaping, reparsing, then is it doing what you want, etc. I've been using LevelDB (via plyvel) a lot lately for data storage, and every time I end up having to use SQL (even indirectly via ORM) is painful by comparison because you can just feel the control being taken away from you, and somewhere you end up having to fire up the DB prompt for no good reason such as adding strange DB specific indexing flags to columns or even setting up authorization and DB creation, making it yet another thing to go wrong during deployment. Protocol buffers stored in LevelDB prove so easy to use by comparison with something like SQLite or PgSQL just at an API level. The resulting code is simpler, cleaner, and much easier to reason about. If I need to move it to another format the code, again, is amazingly small. As someone that cares deeply about my app's data structure seeing the acceptance of a world beyond SQL is one of the best developments in my career, and experience means I simply don't trust any SQL based abstraction to give you the controls to get it right.

coldtea 13y ago
>I'm totally the opposite, and blame you entirely for provoking what follows! SQL at this point is passed expiry. Actually it's the very opposite. SQL is a higher level abstraction for all this ad-hoc junk, based on actual mathematical principles (relational algebra), and it's also declarative, instead of imperative. Furthermore, all this NoSQL query systems now emerging are nothing new. They were tried in the 70s and early 80s, and people found out that they sucked. Before SQL what we had was, well, NoSQL. Wanting a flexible schema-less db or one that's denormalised for certain needs (like Google's or Facebook's) makes sense. Replacing SQL and RBDMS for the common tasks they are used (company logistics and accounting, etc) with NoSQL is a regression to the primitive past. I guess people not knowning (computing) history are doomed to repeat it.

baruch 13y ago

I much prefer ReQL over SQL. It is much nicer, flows naturally in my opinion and is combine-able. I'm using this combination method in my rethink-miner tool to help me work with data to try and figure it out and I can make generic queries that I can use on top of every other sub-query which is very very nice.

j_s 13y ago

For now it probably keeps a number of users away that they don't want to support, but it makes sense to at least have an automatic translation layer in the long run.

chipotle_coyote 13y ago

I'm not really sure how you could wedge SQL into a system that doesn't really have the concept of columns without it being, at the least, a distinctly different dialect. And, really, if you've worked with some of the "higher level" interfaces to SQL like SQLAlchemy or Laravel's QueryBuilder, ReQL doesn't seem that alien; they all tend to work by method chaining as well. (But they usually don't support everything SQL does, unlike ReQL's interfaces.) Also: unlike my (admittedly limited) exposure to other NoSQL systems, RethinkDB supports relations and joins -- it's actually a lot easier for an SQL fiend to move to Rethink than any of its competitors.