4 ms·
Mike Stonebraker: The "NoSQL" Discussion has Nothing to Do With SQL
- petewarden 17y agoMy summary: it's possible to jettison a lot of ACID requirements and implement sharding and still have SQL as the interface to your data. My response: Sure, in theory, but for right now Redis/Tokyo Tyrant/etc actually exist and are free. They don't support a lot of what traditional databases consider required features, but the NoSQL movement is based around a recognition that many applications can sacrifice those in favor of performance and scalability. Key/value stores are like C, barely disguised assembler that lets you shoot yourself in the foot, but is incredibly flexible. NoSQL is about acknowledging that sometimes that's the right tool for the job. This article doesn't address that choice at all, just handwaves about the wonderful technology that will handle all the problems automagically Real Soon Now. Point me at an SQL database I can use to handle my data workload of thousands of updates a second on a cheapie EC2 instance, that will be a discussion I can use.
- strlen 17y agoThere are also SQL based non-relational, non-acid, non (strictly speaking) database systems too, particularly for OLAP uses. Hive (and in the most recent versions/through a plugin, Pig too) provides SQL support on top of Hadoop. While that's somewhat of a misplaced fit (SQL is based on relational algebra, why use relational algebra on a non-relational store?), it does provide a convenient interface for ad-hoc querying, particularly for non-engineers (e.g. business analysts). Edit: addendum - Honestly, my own personal preference would be with either a language specifically designed for the non-relational store in question (e.g. Pig Latin) or a DSL in a general purpose programming language (e.g. Cascading or especially Clojure-based macros the Flightcaster team has created on top of Cascading).
- neilc 17y agoSure, in theory, but for right now Redis/Tokyo Tyrant/etc actually exist and are free. Yes, there's a question of whether we are talking the right way to build systems, or which system you ought to use in production tomorrow. Building a whole "movement" around the simple lack of some software is a little short-sighted, IMHO. BTW, Stonebraker's VoltDB (www.voltdb.com) is a high-performance OLTP engine, and apparently an alpha release will be open-sourced shortly (a few months, I believe). the NoSQL movement is based around a recognition that many applications can sacrifice those in favor of performance and scalability. You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time.
- jerf 17y ago"You make it sound like the NoSQL people were the first to observe this. That is very far from the truth -- IMS, Codasyl, Berkeley DB, object-oriented DBs, etc. have all been around for a long time." Yes. The whole "NoSQL movement" doesn't make any sense until you recognize it not as a brilliant technical development, but as a backlash against hordes of people who always answered "What is the best way to store X?" with loud shouts of "SQL! And you're a moron if you disagree!" without ever examining the nature of X. (Yeah, I'm obviously exaggerating. But only a bit. Don't even pretend that there haven't been people running around and shouting this at every available opportunity.) "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all, end-all answer to every problem ever. In a technical sense they're not even remotely new; what's new is the cracking of the SQL dogma in common perception, brought on by an increasing number of workloads that SQL databases just can't handle economically. (Which is to say, even if there exists a SQL DB and a DB server that may meet some need, SQL doesn't win if the server is actually more expensive than using a "NoSQL" solution.)
- gaius 17y ago"SQL! And you're a moron if you disagree!" No-one ever said this, because no-one who actually knows anything about databases would think you store anything with a query language! It makes as much sense as answering "what filesystem should we use?" with "C!". "NoSQL" is ultimately more about the observation that relational databases indeed are not the be-all No, it's about thinking that MySQL is the be-all and end-all of relational database technology, whereas in feature terms it's 15-20 years behind the major players (indeed, only has basic functionality thanks to one of those players). When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't.
- jerf 17y agoActually, there is a subtlety you've missed. We don't use relational databases. We use SQL, and endless variants thereof. I'd be a lot less grumpy about relational databases if there was a relational-but-not-SQL way of using them. Thought this grumpiness has nothing to do with the "NoSQL" issues, I just hate being stuck on 1970s syntax. (No, this is not a complaint about "sets". It is in fact the opposite, that it doesn't support the truly relational style of programming anywhere near well enough!) "When they come out with statements like "SQL doesn't scale" I look at RDBMSs storing >100T of data and/or handling >10,000 COMMITs/sec and I very quickly figure out who has actual experience and who doesn't." And you're gliding right over the question of what is being stored, how it is being accessed, on what timeframe it is being accessed, what is being done with it, and how much it costs vs other options which is something I rather made a point of pointing out. A $10 million dollar solution to something that could have been solved with $0 and five hundred developer hours is not an acceptable solution. The fact that the expensive solution works isn't even relevant, really. You know, all those pesky engineering details that SQL, sorry, RELATIONAL-uber-alles advocates always conveniently forget about. Nothing is free.
- bham 17y agoPlease refer to this independent and unbiased study that bolsters my point, a study which I authored.
- gcheong 17y agoI see nothing wrong in citing your own work. He does make a disclaimer at the end that he is involved in several RDBMS technology startups.
- gord 17y agoThe article starts with a description/definition of 'key-value stores' vs 'document stores'... but doesn't clarify the difference, if any. When people say 'NoSQL', that doesn't preclude another kind of query language... I assume NoSQL to mean, roughly : - no traditional ansi RDBMS SQL - key-value store - graph-like / treelike data as first class citizen - map/reduce style operations - bundled with a dynamic lisp/python/ruby/javascript style general programming language. You'd have to agree that 'NoSQL' is a movement - ie. many people vocalizing their shared emotional frustration at the limits of SQL. Not theoretical limits, but practical ease of use and subjective syntactic inelegance. I personally identify with the movement, after having built traditional sql systems, preached data normalization etc., then slowly realising that this RDB approach is just really unwieldy and brittle. Moving from XML to JSON, from C++ to lisp languages.. this makes the ugly syntax of SQL really stand out : as blatantly as COBOL or FORTRAN do to someone who has read K&R. Maybe we havent found the right replacement... but we still know there has to be a better way. The term 'NoSQL' is subjective, undefined, fuzzy - but its a label for a real problem that needs solving, and groups together partial solutions.
- vicaya 17y agoActually many NoSQL DBs are not just about relaxing ACID requirements, but also about limiting functionality for easier scaling via DHT. They only support key/value set/get and not range scan, which is needed to implement SQL semantics. I think the best interpretation for NoSQL so far is "Not only SQL", which has interface implication as well.
- nimbix 17y agoI think it's only a matter of time before this NoSQL movement morphs into a SPARQL movement. http://en.wikipedia.org/wiki/SPARQL http://en.wikipedia.org/wiki/SPARQL
- kevinpet 17y agoThe title is correct, the article confused and somewhat pointless. NoSQL is about the systems, not the query language, but it's not about RDBMS minus atomicity and consistency, it's about saying "this not SQL, don't compare it to RDBMS that you are used to querying with SQL". Anyone talking about how you can implement SQL on top of NoSQL systems is missing the point. KV and Document stores are a different way of storing data. SQL: express what you want to happen using relational notation, and it will magically happen. There are no real-world performance issues. NoSQL: certain operations are fast, other operations are slow. This fact is reflected in the API.
- jsrn 17y ago> SQL: express what you want to happen using relational notation, > and it will magically happen. There are no real-world > performance issues. I don't think that's true: Consider the CREATE INDEX statement - it's part of the API and it is intended for improving performance. True, a RDBMS will execute a query even if the programmer didn't issue CREATE INDEX before - but this more reflects the rigid separation ('data independence') of the logical and physical data model than a lack of control (API exposure) over performance details.
- chwahoo 17y agoThe article isn't confused - he's using the SQL in the same way you are (in your first paragraph, anyway) - he's referring to RDBMS's. His point is that that many of the performance benefits of a KV/documentstore based db can be achieved in an RDBMS without throwing away time-tested ideas like ACID guarantees. I was confused by your post, however. You initially state that NoSQL is about the system/approach to storing data (and not the query language) but then proceed to contrast querying APIs.
- alecco 17y agoGuys, please check his Wikipedia page before starting some random rambling. This guy is a living legend and has several of the most prestigious awards. He also is/was an entrepreneur and large DB corp CTO. Oh, and a very early open source advocate and developer. Among many other things. Don't say something stupid just because you like NoSQL. In fact, he was an early and strong critic of the one-size-fits-all commercial RDBMS solutions. [...] These papers presented reasons and experimental evidence that showed that the major RDBMS vendors can be outperformed by 1-2 orders of magnitude by specialized engines in the data warehouse, stream processing, text, and scientific database markets. 1-2 orders of magnitude, that is in the range of 2-99 times faster. And in particular, if you didn't read at least 5 of his papers you can't pretend to have a clue about how database engines, and in particular the major RDBMS, work. Thanks. [A n00b DB engine writer, currently working on a non-SQL database] PS: If I get to achieve 10% of what he did so far in his life I'd consider my life amazing. And he is still making new things. Respect!
- antirez 17y agoI'm a bit shocked by your post. So because he is a living legend one should not say nothing before reading all the papers cited even if from the article his point is already clear and one may say something meaningful even as a modest developer? I'm going to say one for instance: the article's main point is about the fact that because of a threaded and disk-based architecture you can't get much faster than well engineered SQL architectures. For instance Redis is completely excluded from this reasoning, being in-memory and single-threaded. Another point: where are the numbers? For instance I can trivially give proof that Redis can handle 150,000 operations/second per core in a decent Linux Box. I want to know what's the performance of the SQL solutions cited in the article. Another one: all the SQL solutions cited in the article are names I never heard before of today. NoSQL DBs are tar.gz you can download for free from sites and compile now without paying nothing. Respect does not mean to shut up.
- cx01 17y ago> the article's main point is about the fact that because of a threaded and disk-based architecture you can't get much faster than well engineered SQL architectures. I disagree. His main point is that current SQL databases are slow not because they use SQL but because their implementations suck. He then lists 4 areas where the implementations spend most of the time and argues that all of them can be eliminated (how this should be done is explained in the linked paper).
- jvyduna 17y agoIndeed Stonebreaker is a living legend. Entrepreneurs may find it interesting that he is currently pitching VCs a commercial venture to augment his latest DB project, http://scidb.org/ http://scidb.org/ (much like Cloudant is to Couch)
- antirez 17y agoI also found this: "VoltDB is an independent, Boston area-based company co-founded by DBMS R&D pioneer Mike Stonebraker and startup veteran Andy Palmer, working in stealth mode on the next-generation of OLTP DBMS." So it's a biased legend. Sincerely if one is investing a lot in something like VoltDB this buzz about NoSQL DBs and especially the fact that they are starting to be actually used to get work done can be a problem.
- jasonwatkinspdx 17y agoI believe this is the commercialization of H-Store, which has been featured here on HN before.
- igrekel 17y agoI am much more interested in the upcoming post. Even tough I have not used the products mentioned. I have seen that RDBMSes can be configured and optimized but usually it costs time or money plus it can get quite complex. This complexity wouldn't be so bad if it allowed you to make the rest of your application simpler. Sadly, it usually isn't the case. You end up constantly dealing with theses two different worlds in your application and ORMs also bring their share of complexity when you do more sophisticated or customized things. And I am not even saying anything about when you need to change things. I think one of the nice things about many of the "noSQL" solutions is that they keep things simple and under your control. You still need to do the complex stuff yourself, but its never simple anyway. I am sure they don't bring a solution to everything but it certainly is nice to see the area of databases moving again after many years of status quo. And the author is certainly one of the people pushing the field.
- nullexpulsion 17y agoapparently, as a supporter of the NoSQL movement, i would just like to state a small 'analogy'... Imagine if someone from amongst our early ancestors hadn’t thought of creating the wheel, where would we be? And so it is with NoSQL - it is just an alternative way of looking at a problem which in this case is that of databases and looks at it in terms of a non-relational model. Therefore, i would like to urge all who are 'hostile' towards it - just let us do what we want. If the results are not to your liking what can we do. i personally believe that through pursuing the principles on which NoSQL is based we will manage to achieve something new that mayhap replace SQL. Additionally, please do also give thought to the fact that Google's Big Table, Amazon's Dynamo and Cassandra can be thought of as part of NoSQl movement.. Rebels are at first turned away from society in general - but later on society realises that the rebels had the correct idea. Change is always essential to progress without which i would not have been typing this.. On another account (sorry for the detour) i am rather getting quite interested in Gopher these days. Anybody interested in Gopher? [ I apologise for the non-tech response.] [ Null Expulsion - nullexpulsion [at] gmail [dot] com - how long will it be since spammers figure out how to detour around this? ]