4 ms·

My summary: it's possible to jettison a lot of ACID requirements and implement sharding and still have SQL as the interface to your data. My response: Sure, in

My summary: it's possible to jettison a lot of ACID requirements and implement sharding and still have SQL as the interface to your data. My response: Sure, in theory, but for right now Redis/Tokyo Tyrant/etc actually exist and are free. They don't support a lot of what traditional databases consider required features, but the NoSQL movement is based around a recognition that many applications can sacrifice those in favor of performance and scalability. Key/value stores are like C, barely disguised assembler that lets you shoot yourself in the foot, but is incredibly flexible. NoSQL is about acknowledging that sometimes that's the right tool for the job. This article doesn't address that choice at all, just handwaves about the wonderful technology that will handle all the problems automagically Real Soon Now. Point me at an SQL database I can use to handle my data workload of thousands of updates a second on a cheapie EC2 instance, that will be a discussion I can use.

strlen 17y ago
There are also SQL based non-relational, non-acid, non (strictly speaking) database systems too, particularly for OLAP uses. Hive (and in the most recent versions/through a plugin, Pig too) provides SQL support on top of Hadoop. While that's somewhat of a misplaced fit (SQL is based on relational algebra, why use relational algebra on a non-relational store?), it does provide a convenient interface for ad-hoc querying, particularly for non-engineers (e.g. business analysts). Edit: addendum - Honestly, my own personal preference would be with either a language specifically designed for the non-relational store in question (e.g. Pig Latin) or a DSL in a general purpose programming language (e.g. Cascading or especially Clojure-based macros the Flightcaster team has created on top of Cascading).
neilc 17y ago
Sure, in theory, but for right now Redis/Tokyo Tyrant/etc actually exist and are free. Yes, there's a question of whether we are talking the right way to build systems, or which system you ought to use in production tomorrow. Building a whole "movement" around the simple lack of some software is a little short-sighted, IMHO. BTW, Stonebraker's VoltDB (www.voltdb.com) is a high-performance OLTP engine, and apparently an alpha release will be open-sourced shortly (a few months, I believe). the NoSQL movement is based around a recognition that many applications can sacrifice those in favor of performance and scalability. You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time.
- jerf 17y ago
  "You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time." Yes. The whole "NoSQL movement" doesn't make any sense until you recognize it not as a brilliant technical development, but as a backlash against hordes of people who always answered "What is the best way to store X?" with loud shouts of "SQL! And you're a moron if you disagree!" without ever examining the nature of X. (Yeah, I'm obviously exaggerating. But only a bit. Don't even pretend that there haven't been people running around and shouting this at every available opportunity.) "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all, end-all answer to every problem ever. In a technical sense they're not even remotely new; what's new is the cracking of the SQL dogma in common perception, brought on by an increasing number of workloads that SQL databases just can't handle economically. (Which is to say, even if there exists a SQL DB and a DB server that may meet some need, SQL doesn't win if the server is actually more expensive than using a "NoSQL" solution.)
  - gaius 17y ago
    "SQL! And you're a moron if you disagree!" No-one ever said this, because no-one who actually knows anything about databases would think you store anything with a query language! It makes as much sense as answering "what filesystem should we use?" with "C!". "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all No, it's about thinking that MySQL is the be-all and end-all of relational database technology, whereas in feature terms it's 15-20 years behind the major players (indeed, only has basic functionality thanks to one of those players). When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't.
    - jerf 17y ago
      Actually, there is a subtlety you've missed. We don't use relational databases. We use SQL, and endless variants thereof. I'd be a lot less grumpy about relational databases if there was a relational-but-not-SQL way of using them. Thought this grumpiness has nothing to do with the "NoSQL" issues, I just hate being stuck on 1970s syntax. (No, this is not a complaint about "sets". It is in fact the opposite, that it doesn't support the truly relational style of programming anywhere near well enough!) "When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't." And you're gliding right over the question of what is being stored, how it is being accessed, on what timeframe it is being accessed, what is being done with it, and how much it costs vs other options which is something I rather made a point of pointing out. A $10 million dollar solution to something that could have been solved with $0 and five hundred developer hours is not an acceptable solution. The fact that the expensive solution works isn't even relevant, really. You know, all those pesky engineering details that SQL, sorry, RELATIONAL-uber-alles advocates always conveniently forget about. Nothing is free.

strlen 17y ago

There are also SQL based non-relational, non-acid, non (strictly speaking) database systems too, particularly for OLAP uses. Hive (and in the most recent versions/through a plugin, Pig too) provides SQL support on top of Hadoop. While that's somewhat of a misplaced fit (SQL is based on relational algebra, why use relational algebra on a non-relational store?), it does provide a convenient interface for ad-hoc querying, particularly for non-engineers (e.g. business analysts). Edit: addendum - Honestly, my own personal preference would be with either a language specifically designed for the non-relational store in question (e.g. Pig Latin) or a DSL in a general purpose programming language (e.g. Cascading or especially Clojure-based macros the Flightcaster team has created on top of Cascading).

neilc 17y ago

Sure, in theory, but for right now Redis/Tokyo Tyrant/etc actually exist and are free. Yes, there's a question of whether we are talking the right way to build systems, or which system you ought to use in production tomorrow. Building a whole "movement" around the simple lack of some software is a little short-sighted, IMHO. BTW, Stonebraker's VoltDB (www.voltdb.com) is a high-performance OLTP engine, and apparently an alpha release will be open-sourced shortly (a few months, I believe). the NoSQL movement is based around a recognition that many applications can sacrifice those in favor of performance and scalability. You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time.

jerf 17y ago
"You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time." Yes. The whole "NoSQL movement" doesn't make any sense until you recognize it not as a brilliant technical development, but as a backlash against hordes of people who always answered "What is the best way to store X?" with loud shouts of "SQL! And you're a moron if you disagree!" without ever examining the nature of X. (Yeah, I'm obviously exaggerating. But only a bit. Don't even pretend that there haven't been people running around and shouting this at every available opportunity.) "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all, end-all answer to every problem ever. In a technical sense they're not even remotely new; what's new is the cracking of the SQL dogma in common perception, brought on by an increasing number of workloads that SQL databases just can't handle economically. (Which is to say, even if there exists a SQL DB and a DB server that may meet some need, SQL doesn't win if the server is actually more expensive than using a "NoSQL" solution.)
- gaius 17y ago
  "SQL! And you're a moron if you disagree!" No-one ever said this, because no-one who actually knows anything about databases would think you store anything with a query language! It makes as much sense as answering "what filesystem should we use?" with "C!". "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all No, it's about thinking that MySQL is the be-all and end-all of relational database technology, whereas in feature terms it's 15-20 years behind the major players (indeed, only has basic functionality thanks to one of those players). When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't.
  - jerf 17y ago
    Actually, there is a subtlety you've missed. We don't use relational databases. We use SQL, and endless variants thereof. I'd be a lot less grumpy about relational databases if there was a relational-but-not-SQL way of using them. Thought this grumpiness has nothing to do with the "NoSQL" issues, I just hate being stuck on 1970s syntax. (No, this is not a complaint about "sets". It is in fact the opposite, that it doesn't support the truly relational style of programming anywhere near well enough!) "When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't." And you're gliding right over the question of what is being stored, how it is being accessed, on what timeframe it is being accessed, what is being done with it, and how much it costs vs other options which is something I rather made a point of pointing out. A $10 million dollar solution to something that could have been solved with $0 and five hundred developer hours is not an acceptable solution. The fact that the expensive solution works isn't even relevant, really. You know, all those pesky engineering details that SQL, sorry, RELATIONAL-uber-alles advocates always conveniently forget about. Nothing is free.