7 ms·

Cassandra vs MongoDB vs CouchDB vs Redis vs Riak comparison

380 points by kkovacs 16y ago

arethuza 16y ago
My understanding is that in CouchDB you can't guarantee that older versions of documents will still exists (they might be there, but they could have been removed by compaction or not replicated). However, there is a fairly nice way of storing older versions of documents - hold older versions as file attachments on the document. See: http://jchrisa.net/drl/_design/sofa/_list/post/post-page?startkey=[%22Versioning-docs-in-CouchDB%22] http://jchrisa.net/drl/_design/sofa/_list/post/post-page?sta...
- jchrisa 16y ago
  thanks. latest version of the article is here now http://blog.couchone.com/post/632718824/simple-document-versioning-with-couchdb http://blog.couchone.com/post/632718824/simple-document-vers...
  - kkovacs 16y ago
    THAT jchrisa, seriously? Wow, I'm honored that you chimed in! Thanks, man :)
    - jchrisa 16y ago
      Thanks for the solid comparison, it's a breath of fresh air. We're working on the mobile platform SDKs right now, should be rolling them out all through 2011.
- kkovacs 16y ago
  Yes, if versioning is the main point, then surely one should take special measures to keep them, as in the case of invoices (stornoed, stornos stornoed, etc). As for a CMS system, for example, it's nice to have most of the history, but nobody gets hurt if some gets lost during replication outages, etc. Just my $0.02 :) K.
- iamwil 16y ago
  This is probably the biggest misunderstanding of couchdb, imo. The versioning system in couchdb is only there to make the seamless replication possible. There's no guarantee that previous versions will exist at a future time, like in git. Where couchdb has some immense possibilities is in distributed applications, not only server side, but also mobile phones and browsers. Since you can write and contain an entire webapp inside of couchdb, you can technically replicate the entire app to your mobile phone, and it'd work offline or online. And if you need your app on another platform--as long as it has couchdb, you can just replicate it there. I never see this mentioned in any overviews of comparison for couchdb. The sticking point right now, though, is that couchdb isn't on very many mobile platforms. There has only been experiments with writing couchdb on top of HTML5's localStore, and jChris et al are working on Couchdb for android.
  - kkovacs 16y ago
    I didn't mention that (it does not really matter when choosing one right now), but I too consider that one of the most exciting explorations in this area! Keep up the good work!
  - bitdiddle 16y ago
    "The versioning system in couchdb is only there.." CouchDB does not version, period.
    - Periodic 16y ago
      The "versioning" is really just there to support their optimistic concurrency model, if I recall. The idea is that you know you need to retry your operation if the version hash of the file has gone up since you last read the data and thus you know your local file is out of date. As I recall, the id field is just a string. It's just common to let it do the automatic "#-hash" representation. It's been a while since I played with CouchDB though, so I could be off.
      - dhimes 16y ago
        You are correct.
- SoftwareMaven 16y ago
  Trying to use CouchDB's versioning system seems like depending on a very leaky abstraction. That you can see it's versioning is a side-effect of how it behaves; not a feature it is providing. If you need versioned records, you are likely better off identifying your versioning requirements and building to those than trying to piggyback off of something else poorly suited.
mikeytown2 16y ago
I was hoping HandlerSocket would be in here. If you don't know about it, check it out http://news.ycombinator.com/item?id=1886137 http://news.ycombinator.com/item?id=1886137
- kkovacs 16y ago
  I don't know that one yet, but I promise I'll read up on it! Thanks! K.
  - mikeytown2 16y ago
    The latest Percona flavor of MySQL has it built in http://www.mysqlperformanceblog.com/2010/12/14/percona-server-now-both-sql-and-nosql/ http://www.mysqlperformanceblog.com/2010/12/14/percona-serve...
deleted 16y ago
[deleted]
- kkovacs 16y ago
  I meant database consistency in the sense of never needing "repair tables"; although as far as I know, CouchDB's automatic conflict resolution is also consistent (based on UUIDs), isn't it? Clarified it, thanks!
  - deleted 16y ago
    [deleted]
    - kkovacs 16y ago
      Thanks Sean, I'll read up on that. I have experience with CouchDB only under normal loads -- our heavy-load stuff (stock data) uses Redis. :)
    - davisp 16y ago
      They're not guaranteed to be the primary revision for a document (ie, the last write could be listed as the conflict) but the data is still there so that client code can resolve the conflict. Also of note, if the edits that caused the conflict are replicated to other nodes, each node will independently choose the same revision to use as the 'primary' document response. Bottom line, the choice is deterministic and is guaranteed to be preserved, but the choice may not be the last written revision. Also, bugs that result in corrupt dbs are treated as major bugs as opposed to a part of the design. I've also not seen reports of index corruption under load, if you have logs or any more information we'd definitely appreciate if you could put that info into a ticket on JIRA [1] or even just mail dev@couchdb.apache.org with details. [1] https://issues.apache.org/jira/browse/COUCHDB https://issues.apache.org/jira/browse/COUCHDB
- jchrisa 16y ago
  Couch actually uses a first-write wins model (MVCC optimistic concurrency) not a last-write-wins model. The difference between those two is huge.
ocharles 16y ago
> While SQL databases are insanely useful tools, their tyranny of ~15 years is coming to an end This shit, AGAIN? Really? No, they are not.
- kkovacs 16y ago
  Not that they get thrown away, they are still very useful. But at least there's an alternative now. I mean, seriously, Zabbix keeping monitoring data in MySQL? Also Piwik? That's a sick solution, IMHO. :) (BTW, I love Zabbix and Piwik. Use them both. Only I think that having no good alternate data store at the time of their writing, their data storage is suboptimal.)
- va_coder 16y ago
  With AppEngine at Google, MongoDB at Disqus, Cassandra at Facebook and Redis at Github you can definitely say that SQL databases are one of many options available today and don't dominate like they did 5 years ago.
  - dennyabraham 16y ago
    if i'm not mistaken, the majority of those organizations still rely heavily on relational datastores, except in the case of exceptional workloads. in addition, i believe facebook has since migrated away from cassandra to the hadoop stack for their messaging platform, though they primarily use mysql (or its successors). SQL is being replaced in niches that strain its model. elsewhere, it remains steadfast.
    - kkovacs 16y ago
      I agree, ultimately it comes to everyone's own definition of "tyranny". :) I meant it as "not really having any defensible other choice". (While, of course for example CDB and BerkleyDB have "always" existed.) I think nobody expects SQL's "market share" to fall to low levels, especially with noSQL requiring much deeper understanding of the data and it's planned use. NoSQL practically operates on a lower layer than SQL does. Still, it's nice to see people thinking about data storage choices and not going blindly to MySQL/Oracle/etc!
schmichael 16y ago
Under protocols you may want to specify MongoDB's as BSON and Cassandra's as Thrift. That would be more helpful than "binary/custom". Updated: Also Redis's main selling point is it's extensive data structure/operations support. "Blazingly fast" really depends on what your workload is and what you're comparing it against.
- kkovacs 16y ago
  Protocols: great idea, thanks man, amended it! Blazing fast: I mean compared to the other four.
  - schmichael 16y ago
    I've had both MongoDB and Cassandra perform nearly as well as Memcache when getting a single document/row when the document was in memory in MongoDB and the row was cached (row cache, not just key cache, so again: fully in memory) in Cassandra. In memory operations are fast in many databases. Redis's default configuration (vm-enabled no) just only does operations in memory (with an occasional sync to disk). That's terrible durability but fantastic performance. Most databases, including Redis, can be configured for either that sort of high-performance/low-durability or the opposite. It's just that their default settings/behaviors vary widely.
  - jfb 16y ago
    Sorted sets. I mean, c'mon. That's worth the price of admission right there.
- roder 16y ago
  Riak also supports, in addition to HTTP, Protocol Buffers - a binary encoding protocol introduced by Google. A few other notable Riak features include a JS or Erlang MapReduce API and full-text search.
  - kkovacs 16y ago
    Thanks, amended RIAK with Erlang map/reduce and full-text search! (I need to read up on Protocol Buffers, but it sounds useful, too). (Updated: typo)
littleidea 16y ago
Apples vs Oranges vs Strawberries vs Pineapple vs Grapes Apples usually stay crispy unless baked. Good in pies. Oranges can be sour (or sweet). Do not bake. Strawberries are red. Good in pies, advise against baking. Pineapples are rough on the outside. Good fresh, baked, grilled, fried, debatable on pizza. Grapes come in many colors and sizes. Great fresh or turned into alcoholic beverages. (Not the worst introduction to fruit, but perhaps superficial? Amirite?)
- kkovacs 16y ago
  Perhaps it is. I still hope that it's useful :)
  - littleidea 16y ago
    I think it is useful as a catalog of properties, I just think calling it a comparison or using 'vs' is a stretch. Proverbial Apples to Oranges problem
- gokhan 16y ago
  So it seems it's quite valid to compare them in terms of baking, as ingredients for pies etc.
- taylorbuley 16y ago
  I'd rather know about these superficialities than know nothing at all.
  - hendler 16y ago
    I agree. And I think superficiality is different than generality or summary. It's a summary. And I appreciate the effort of some one taking the time to put their work online.
    - kkovacs 16y ago
      Thanks, I appreciate the kind words! K.
waratuman 16y ago
You mention that some of these solution could be used in the Financial industry. I would be cautious of using these, especially since some are eventually consistent. If you are just tracking data these may be fine though.
- kkovacs 16y ago
  I agree with that, but you know, it's very hard to find good examples. :) I'm open to better suggestions :)
  - hendler 16y ago
    Some points on Cassandra: - Facebook designed it for inbox feature - - SOLR/Lucene is being integrated - recently Sequoia backed Riptano - see http://www.riptano.com/ http://www.riptano.com/
    - mjphilli 16y ago
      But facebook recently dumped it for inbox feature and began using Hbase instead. Not sure if Sequoia backing Riptano is supposed to be a bug or a feature?
- rbranson 16y ago
  Most of the financial sector is eventually consistent. I'm not just talking about traditional banks, I'm also talking about the markets as well. Most revolve around batch settlement processes where consistency essentially occurs on a schedule. While at the micro level, the components of these systems are somewhat built on ACID databases, that becomes irrelevant when as a system, it functions as eventually consistent. Dynamo itself is based on the fully ACID BerkeleyDB at the node level.
ares2012 16y ago
I'm curious why you wouldn't include HBase as it's the dominant solution for NoSQL in systems requiring data consistency?
- kkovacs 16y ago
  Only beacuse I don't know HBase very well. I'll read up on it, at least try it, and then add that too (and also maybe Tokyo Cabinet). Thanks for the feedback!
  - muloka 16y ago
    Cool... and maybe also Voldemort?
  - dhimes 16y ago
    Looks like you fixed this. Thanks!
lukev 16y ago
Interesting and useful. One major feature differentiator is something it doesn't really talk about, though - how conducive is each system to Massive Data? For example, he kind of has a bone to pick with Cassandra, which is probably justified. But from what little I know, one of the features of Cassandra is that it's designed to scale pretty much to infinity. That may be true of a couple of the others, but for some (like CouchDB) it isn't a design goal at all.
- kkovacs 16y ago
  Good point, and it's not there since I only wanted to speak from experience; especially with rumors of Cassandra scaling problems at Reddit and Digg. But sure thing, "infinite" scaling is probably best done with the Dynamo-like stuff like Cassandra and/or RIAK.
  - DennisP 16y ago
    Reddit runs Cassandra with just a few nodes. Cassandra scales up well, but doesn't scale down as well: http://www.reddit.com/r/announcements/comments/c2spc/reddits_may_2010_state_of_the_servers_report_or/c0ptl28 http://www.reddit.com/r/announcements/comments/c2spc/reddits...
- bitdiddle 16y ago
  "That may be true of a couple of the others, but for some (like CouchDB) it isn't a design goal at all." actually it is, you might take a peek at BigCouch[1], it puts the C in CouchDB. [1] https://github.com/cloudant/bigcouch https://github.com/cloudant/bigcouch
ghshephard 16y ago
Worth adding HBase? Much below Stolen from their overview page (All needs to be confirmed): http://hbase.apache.org/ http://hbase.apache.org/ WRITTEN IN: Java MAIN POINT: Hadoop Database LICENSE: Apache PROTOCOL: A REST-ful Web service gateway This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Hadoop. HBase includes: Convenient base classes for backing Hadoop MapReduce jobs with HBase tables Query predicate push down via server side scan and get filters Optimizations for real time queries A high performance Thrift gateway A REST-ful Web service gateway that supports XML, Protobuf, and binary data encoding options Cascading, hive, and pig source and sink modules Extensible jruby-based (JIRB) shell Support for exporting metrics via the Hadoop metrics subsystem to files or Ganglia; or via JMX HBase 0.20 has greatly improved on its predecessors: No HBase single point of failure Rolling restart for configuration changes and minor upgrades Random access performance on par with open source relational databases such as MySQL FOR EXAMPLE: Facebook Messaging Database BEST USE: Use it when you need random, realtime read/write access to your Big Data.
- kkovacs 16y ago
  Thanks, Gordon, I put it in (shortened some lines). It would be great to have a "more general" for-example, since noone outside Facebook meets the problem of "let's build Facebook's messaging database" :) Any suggestions?
  - sigzero 16y ago
    There might be something you can glean from: http://wiki.apache.org/hadoop/Hbase/PoweredBy http://wiki.apache.org/hadoop/Hbase/PoweredBy
- yummyfajitas 16y ago
  I'll question you on one point: "Use it when you need random, realtime read/write access to your Big Data." Does HBase now do a good job of random access? I was always under the impression that it did random access adequately, but it's real strength was with scans (based on ordering of keys).
fjabre 16y ago
Membase? Using it in a recent project and it's been working great for us.
- kkovacs 16y ago
  Seems like an interesting project too, will try it! Thanks!
- mjphilli 16y ago
  Membase is the one to watch. I saw a presentation by AOL @ Hadoop World re: their use of Membase. Incredibly impressive technology.
antirez 16y ago
I like this article: while it is for sure not the definitive guide to NoSQL, it is a short description mostly about facts that people new to the field can use to get an idea about what a good candidate could be for initial experimentation, given a defined problem to solve. That said I think that picking the good database is something you can do only with a lot of work. Picking good technologies for your project is hard work, so there is to try one, and another and so forth, and even reconsidering after a few years (or months?) the state of the things again, given the evolution speed of the DB panorama in the recent years. While I'm at it I like to share that in this exact days I'm working at a Redis disk back end. I've already a prototype working after a few days of full immersion (I like to use vacation time to work at completely new ideas for Redis). The idea is that everything is stored on disk, in what is a plain key-value database (complex values are serialized when on disk), and the memory is instead used as an object cache. It is like taking current Redis Virtual Memory and inverting the logic completely, the result is the same (working set in memory, the rest on disk), but this implementation means that there are no limits on the data you can put into a single instance, that you don't have slow restarts (data is not loaded on memory if not demanded), and there isn't to fork() to save. Keys marked as "dirty" (modified) are transfered to disk asynchronously as needed, by IO threads. If everything will work as I expect (and initial tests are really encouraging) this means that Redis 2.4 will exit in a few months completely killing the current Virtual Memory implementation in favor of the new "two back ends" design, where you can select if you want to run an in-memory DB or an on-disk DB where memory is just an LRU cache for the working set.
- kkovacs 16y ago
  antirez, it's a honor that you commented, thanks! :) The new inverted logic for the VM you describe seems very interesting; I'm very much looking forward to see 2.4! Redis is already more than perfect what we use it for -- keeping track of stock price data, and distributing it. The size of the DB is known in advance (the amount of stocks does not grow very fast), and the performance is perfect. Keep up the good job! (And have a nice new year) Kristof
  - antirez 16y ago
kkovacs 16y ago
I think it's a nice closing word from @jzy: A SQL query goes into a bar, walks up to two tables and asks, "Can I join you?" "No, but you can enjoy the view." Sorry :) K.
nl 16y ago
CouchDB & MongoDB both share one property that this comparison misses (or mentions only in passing). Both are schema free datastores. For me, this is the biggest, most useful difference between them and traditional SQL databases, because it makes things easy that are very, very hard (or inefficient) on an SQL database. It's probably also worth noting that other NoSQL solutions don't share this advantage. For example, Cassandra requires all nodes to be restarted to apply a schema change, which can be quite a big deal.
- tylerhobbs 16y ago
  "Cassandra requires all nodes to be restarted to apply a schema change, which can be quite a big deal." That's no longer true. In 0.7, keyspaces and column families may be created, altered, or dropped live.
  - nl 16y ago
    I thought it might have been fixed by now. Anyway, you still need a schema with Cassandra.
    - rbranson 16y ago
      Not in the same sense as an SQL database. You can freely add columns and rows, just not Column Families or Keyspaces. This is because a KS+CF combo is stored in it's own file, in a certain order, so that it can be efficiently traversed using natural ordering. If you don't have this need and just need a flat K/V database, you can use a single KS+CF for everything.
markoa 16y ago
What we're missing are similar arricles that go into disadvantages and implications on deployment. Eg I have found out that deploying Tokyo Tyrant in a Rails project requires you to write some sčripts to ensure that things run properly. Also the db size has to be set in configuration in advance. MongoDB OTOH is not designed for a single server environment, has a very small max document size, easily gets corrupted if process is stopped etc.
dfhjtjty 16y ago
Click on our website: === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === "Priority, my friend!! Christmas is coming, quick to our website shopping, our web site shopping there will be something different, unexpected things to you, let you have different sense, our website wholesale various fashion shoes, such as Nike, Jordan, prada, also includes the jeans, shirt, bags, hats and decoration. All these products are our free transport, prices are competitive, we can also accept paypal j, after the payment within short time, can ship. = New era cap $12; Air jordan(1-24)shoes $30; jordan air max oakland raiders $34a€“39; Ed Hardy AF JUICY POLO Bikini $25; Christan Audigier BIKINI JACKET $25; Tshirts (Polo ,ed hardy,lacoste) $15 coogi DG edhardy gucci t-shirts $18; gstar coogi evisu true jeans $35; coach chanel gucci LV handbags $36; Sunglasses(Oakey,coach,gucci,A r m a i n i) $15; === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) ===
deleted 16y ago
[deleted]
redthrowaway 16y ago
So if Cassandra writes are much faster than reads, why would Reddit go that route? Their comment server is consistently breaking on them, and it would seem that a sub-optimal choice of db might be partly to blame.
- cdavid 16y ago
  Because it is normally harder to scale write than read. At least, that's the most obvious thing coming to my mind, I don't know reddit usage.
- rbranson 16y ago
  It's not as lop-sided as this article might have you believe and has largely been mitigated as of late. This is because Cassandra uses read repair, which is a big component of it's strategy to make both reads and writes to scale linearly while also ensuring durability. What is your suggestion otherwise? Any distributed database that is going to be inexpensive, performant, scalable, and durable will need to use some kind of quorum read repair system. Riak, Voldemort, and Dynamo all use read repair with high levels of production success.
sdgherhe 16y ago
-------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- Gold autumn, personality Men’s clothing Shoes, Travel bag that grabs an eye coat Chao packet Free transport -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 --------
benblack 16y ago
This article is mostly marketing phrases from the websites of the various projects. Sadly, much of it is inaccurate, extremely skewed, or otherwise not useful for the stated purpose of comparing the listed databases. For example, CouchDB having a "Main Point" of "DB consistency" might be the case, as it is for Redis, when there is no replication. In replicated configurations, it is definitely not true. Further, the MVCC is weaker in many ways than in a Dynamo system like Riak as you have no way to influence or discover consistency between replicas. I'm sure folks expert in other systems can identify similar errors in the rest of the post. Can someone explain to me who the target audience is for all these NoSQL comparison articles? They are universally poor, yet universally popular.
laran 16y ago
Nice writeup. A good survey of some key tools in the NoSQL space. Thanks!
deleted 16y ago
[deleted]
paxa 16y ago
Also is VertexDB - small graph database. It's written in C, uses Tokyo Cabinet for storing data. Simple http filesystem-like interface. The general advantage - links, that allow to make graph structures on database level. https://github.com/stevedekorte/vertexdb https://github.com/stevedekorte/vertexdb
Juha 16y ago
Thank's for the article, good information. Does anyone have any user amounts about the different no-sql databases? Or just say two most popular ones? I guess some of them will rise above the other's in following years and some will drop. User amounts would indicate which ones have most potential to stay around and be accepted as standard no-sql databases.
luca_garulli 16y ago
Good article, it's a good starting point to let the people to decide where to start in using a NoSQL solution. But what about OrientDB? Do you plan to add it in this feature comparison?

380 points by kkovacs 16y ago

My understanding is that in CouchDB you can't guarantee that older versions of documents will still exists (they might be there, but they could have been removed by compaction or not replicated). However, there is a fairly nice way of storing older versions of documents - hold older versions as file attachments on the document. See: http://jchrisa.net/drl/_design/sofa/_list/post/post-page?startkey=[%22Versioning-docs-in-CouchDB%22] http://jchrisa.net/drl/_design/sofa/_list/post/post-page?sta...

jchrisa 16y ago
thanks. latest version of the article is here now http://blog.couchone.com/post/632718824/simple-document-versioning-with-couchdb http://blog.couchone.com/post/632718824/simple-document-vers...
- kkovacs 16y ago
  THAT jchrisa, seriously? Wow, I'm honored that you chimed in! Thanks, man :)
  - jchrisa 16y ago
    Thanks for the solid comparison, it's a breath of fresh air. We're working on the mobile platform SDKs right now, should be rolling them out all through 2011.
kkovacs 16y ago
Yes, if versioning is the main point, then surely one should take special measures to keep them, as in the case of invoices (stornoed, stornos stornoed, etc). As for a CMS system, for example, it's nice to have most of the history, but nobody gets hurt if some gets lost during replication outages, etc. Just my $0.02 :) K.
iamwil 16y ago
This is probably the biggest misunderstanding of couchdb, imo. The versioning system in couchdb is only there to make the seamless replication possible. There's no guarantee that previous versions will exist at a future time, like in git. Where couchdb has some immense possibilities is in distributed applications, not only server side, but also mobile phones and browsers. Since you can write and contain an entire webapp inside of couchdb, you can technically replicate the entire app to your mobile phone, and it'd work offline or online. And if you need your app on another platform--as long as it has couchdb, you can just replicate it there. I never see this mentioned in any overviews of comparison for couchdb. The sticking point right now, though, is that couchdb isn't on very many mobile platforms. There has only been experiments with writing couchdb on top of HTML5's localStore, and jChris et al are working on Couchdb for android.
- kkovacs 16y ago
  I didn't mention that (it does not really matter when choosing one right now), but I too consider that one of the most exciting explorations in this area! Keep up the good work!
- bitdiddle 16y ago
  "The versioning system in couchdb is only there.." CouchDB does not version, period.
  - Periodic 16y ago
    The "versioning" is really just there to support their optimistic concurrency model, if I recall. The idea is that you know you need to retry your operation if the version hash of the file has gone up since you last read the data and thus you know your local file is out of date. As I recall, the id field is just a string. It's just common to let it do the automatic "#-hash" representation. It's been a while since I played with CouchDB though, so I could be off.
    - dhimes 16y ago
      You are correct.
SoftwareMaven 16y ago
Trying to use CouchDB's versioning system seems like depending on a very leaky abstraction. That you can see it's versioning is a side-effect of how it behaves; not a feature it is providing. If you need versioned records, you are likely better off identifying your versioning requirements and building to those than trying to piggyback off of something else poorly suited.

mikeytown2 16y ago

I was hoping HandlerSocket would be in here. If you don't know about it, check it out http://news.ycombinator.com/item?id=1886137 http://news.ycombinator.com/item?id=1886137

kkovacs 16y ago
I don't know that one yet, but I promise I'll read up on it! Thanks! K.
- mikeytown2 16y ago
  The latest Percona flavor of MySQL has it built in http://www.mysqlperformanceblog.com/2010/12/14/percona-server-now-both-sql-and-nosql/ http://www.mysqlperformanceblog.com/2010/12/14/percona-serve...

deleted 16y ago

[deleted]

kkovacs 16y ago
I meant database consistency in the sense of never needing "repair tables"; although as far as I know, CouchDB's automatic conflict resolution is also consistent (based on UUIDs), isn't it? Clarified it, thanks!
- deleted 16y ago
  [deleted]
  - kkovacs 16y ago
    Thanks Sean, I'll read up on that. I have experience with CouchDB only under normal loads -- our heavy-load stuff (stock data) uses Redis. :)
  - davisp 16y ago
    They're not guaranteed to be the primary revision for a document (ie, the last write could be listed as the conflict) but the data is still there so that client code can resolve the conflict. Also of note, if the edits that caused the conflict are replicated to other nodes, each node will independently choose the same revision to use as the 'primary' document response. Bottom line, the choice is deterministic and is guaranteed to be preserved, but the choice may not be the last written revision. Also, bugs that result in corrupt dbs are treated as major bugs as opposed to a part of the design. I've also not seen reports of index corruption under load, if you have logs or any more information we'd definitely appreciate if you could put that info into a ticket on JIRA [1] or even just mail dev@couchdb.apache.org with details. [1] https://issues.apache.org/jira/browse/COUCHDB https://issues.apache.org/jira/browse/COUCHDB
jchrisa 16y ago
Couch actually uses a first-write wins model (MVCC optimistic concurrency) not a last-write-wins model. The difference between those two is huge.

ocharles 16y ago

> While SQL databases are insanely useful tools, their tyranny of ~15 years is coming to an end This shit, AGAIN? Really? No, they are not.

kkovacs 16y ago
Not that they get thrown away, they are still very useful. But at least there's an alternative now. I mean, seriously, Zabbix keeping monitoring data in MySQL? Also Piwik? That's a sick solution, IMHO. :) (BTW, I love Zabbix and Piwik. Use them both. Only I think that having no good alternate data store at the time of their writing, their data storage is suboptimal.)
va_coder 16y ago
With AppEngine at Google, MongoDB at Disqus, Cassandra at Facebook and Redis at Github you can definitely say that SQL databases are one of many options available today and don't dominate like they did 5 years ago.
- dennyabraham 16y ago
  if i'm not mistaken, the majority of those organizations still rely heavily on relational datastores, except in the case of exceptional workloads. in addition, i believe facebook has since migrated away from cassandra to the hadoop stack for their messaging platform, though they primarily use mysql (or its successors). SQL is being replaced in niches that strain its model. elsewhere, it remains steadfast.
  - kkovacs 16y ago
    I agree, ultimately it comes to everyone's own definition of "tyranny". :) I meant it as "not really having any defensible other choice". (While, of course for example CDB and BerkleyDB have "always" existed.) I think nobody expects SQL's "market share" to fall to low levels, especially with noSQL requiring much deeper understanding of the data and it's planned use. NoSQL practically operates on a lower layer than SQL does. Still, it's nice to see people thinking about data storage choices and not going blindly to MySQL/Oracle/etc!

schmichael 16y ago

Under protocols you may want to specify MongoDB's as BSON and Cassandra's as Thrift. That would be more helpful than "binary/custom". Updated: Also Redis's main selling point is it's extensive data structure/operations support. "Blazingly fast" really depends on what your workload is and what you're comparing it against.

kkovacs 16y ago
Protocols: great idea, thanks man, amended it! Blazing fast: I mean compared to the other four.
- schmichael 16y ago
  I've had both MongoDB and Cassandra perform nearly as well as Memcache when getting a single document/row when the document was in memory in MongoDB and the row was cached (row cache, not just key cache, so again: fully in memory) in Cassandra. In memory operations are fast in many databases. Redis's default configuration (vm-enabled no) just only does operations in memory (with an occasional sync to disk). That's terrible durability but fantastic performance. Most databases, including Redis, can be configured for either that sort of high-performance/low-durability or the opposite. It's just that their default settings/behaviors vary widely.
- jfb 16y ago
  Sorted sets. I mean, c'mon. That's worth the price of admission right there.
roder 16y ago
Riak also supports, in addition to HTTP, Protocol Buffers - a binary encoding protocol introduced by Google. A few other notable Riak features include a JS or Erlang MapReduce API and full-text search.
- kkovacs 16y ago
  Thanks, amended RIAK with Erlang map/reduce and full-text search! (I need to read up on Protocol Buffers, but it sounds useful, too). (Updated: typo)

littleidea 16y ago

Apples vs Oranges vs Strawberries vs Pineapple vs Grapes Apples usually stay crispy unless baked. Good in pies. Oranges can be sour (or sweet). Do not bake. Strawberries are red. Good in pies, advise against baking. Pineapples are rough on the outside. Good fresh, baked, grilled, fried, debatable on pizza. Grapes come in many colors and sizes. Great fresh or turned into alcoholic beverages. (Not the worst introduction to fruit, but perhaps superficial? Amirite?)

kkovacs 16y ago
Perhaps it is. I still hope that it's useful :)
- littleidea 16y ago
  I think it is useful as a catalog of properties, I just think calling it a comparison or using 'vs' is a stretch. Proverbial Apples to Oranges problem
gokhan 16y ago
So it seems it's quite valid to compare them in terms of baking, as ingredients for pies etc.
taylorbuley 16y ago
I'd rather know about these superficialities than know nothing at all.
- hendler 16y ago
  I agree. And I think superficiality is different than generality or summary. It's a summary. And I appreciate the effort of some one taking the time to put their work online.
  - kkovacs 16y ago
    Thanks, I appreciate the kind words! K.

waratuman 16y ago

You mention that some of these solution could be used in the Financial industry. I would be cautious of using these, especially since some are eventually consistent. If you are just tracking data these may be fine though.

kkovacs 16y ago
I agree with that, but you know, it's very hard to find good examples. :) I'm open to better suggestions :)
- hendler 16y ago
  Some points on Cassandra: - Facebook designed it for inbox feature - - SOLR/Lucene is being integrated - recently Sequoia backed Riptano - see http://www.riptano.com/ http://www.riptano.com/
  - mjphilli 16y ago
    But facebook recently dumped it for inbox feature and began using Hbase instead. Not sure if Sequoia backing Riptano is supposed to be a bug or a feature?
rbranson 16y ago
Most of the financial sector is eventually consistent. I'm not just talking about traditional banks, I'm also talking about the markets as well. Most revolve around batch settlement processes where consistency essentially occurs on a schedule. While at the micro level, the components of these systems are somewhat built on ACID databases, that becomes irrelevant when as a system, it functions as eventually consistent. Dynamo itself is based on the fully ACID BerkeleyDB at the node level.

ares2012 16y ago

I'm curious why you wouldn't include HBase as it's the dominant solution for NoSQL in systems requiring data consistency?

kkovacs 16y ago
Only beacuse I don't know HBase very well. I'll read up on it, at least try it, and then add that too (and also maybe Tokyo Cabinet). Thanks for the feedback!
- muloka 16y ago
  Cool... and maybe also Voldemort?
- dhimes 16y ago
  Looks like you fixed this. Thanks!

lukev 16y ago

Interesting and useful. One major feature differentiator is something it doesn't really talk about, though - how conducive is each system to Massive Data? For example, he kind of has a bone to pick with Cassandra, which is probably justified. But from what little I know, one of the features of Cassandra is that it's designed to scale pretty much to infinity. That may be true of a couple of the others, but for some (like CouchDB) it isn't a design goal at all.

kkovacs 16y ago
Good point, and it's not there since I only wanted to speak from experience; especially with rumors of Cassandra scaling problems at Reddit and Digg. But sure thing, "infinite" scaling is probably best done with the Dynamo-like stuff like Cassandra and/or RIAK.
- DennisP 16y ago
  Reddit runs Cassandra with just a few nodes. Cassandra scales up well, but doesn't scale down as well: http://www.reddit.com/r/announcements/comments/c2spc/reddits_may_2010_state_of_the_servers_report_or/c0ptl28 http://www.reddit.com/r/announcements/comments/c2spc/reddits...
bitdiddle 16y ago
"That may be true of a couple of the others, but for some (like CouchDB) it isn't a design goal at all." actually it is, you might take a peek at BigCouch[1], it puts the C in CouchDB. [1] https://github.com/cloudant/bigcouch https://github.com/cloudant/bigcouch

ghshephard 16y ago

Worth adding HBase? Much below Stolen from their overview page (All needs to be confirmed): http://hbase.apache.org/ http://hbase.apache.org/ WRITTEN IN: Java MAIN POINT: Hadoop Database LICENSE: Apache PROTOCOL: A REST-ful Web service gateway This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Hadoop. HBase includes: Convenient base classes for backing Hadoop MapReduce jobs with HBase tables Query predicate push down via server side scan and get filters Optimizations for real time queries A high performance Thrift gateway A REST-ful Web service gateway that supports XML, Protobuf, and binary data encoding options Cascading, hive, and pig source and sink modules Extensible jruby-based (JIRB) shell Support for exporting metrics via the Hadoop metrics subsystem to files or Ganglia; or via JMX HBase 0.20 has greatly improved on its predecessors: No HBase single point of failure Rolling restart for configuration changes and minor upgrades Random access performance on par with open source relational databases such as MySQL FOR EXAMPLE: Facebook Messaging Database BEST USE: Use it when you need random, realtime read/write access to your Big Data.

kkovacs 16y ago
Thanks, Gordon, I put it in (shortened some lines). It would be great to have a "more general" for-example, since noone outside Facebook meets the problem of "let's build Facebook's messaging database" :) Any suggestions?
- sigzero 16y ago
  There might be something you can glean from: http://wiki.apache.org/hadoop/Hbase/PoweredBy http://wiki.apache.org/hadoop/Hbase/PoweredBy
yummyfajitas 16y ago
I'll question you on one point: "Use it when you need random, realtime read/write access to your Big Data." Does HBase now do a good job of random access? I was always under the impression that it did random access adequately, but it's real strength was with scans (based on ordering of keys).

fjabre 16y ago

Membase? Using it in a recent project and it's been working great for us.

kkovacs 16y ago
Seems like an interesting project too, will try it! Thanks!
mjphilli 16y ago
Membase is the one to watch. I saw a presentation by AOL @ Hadoop World re: their use of Membase. Incredibly impressive technology.

antirez 16y ago

I like this article: while it is for sure not the definitive guide to NoSQL, it is a short description mostly about facts that people new to the field can use to get an idea about what a good candidate could be for initial experimentation, given a defined problem to solve. That said I think that picking the good database is something you can do only with a lot of work. Picking good technologies for your project is hard work, so there is to try one, and another and so forth, and even reconsidering after a few years (or months?) the state of the things again, given the evolution speed of the DB panorama in the recent years. While I'm at it I like to share that in this exact days I'm working at a Redis disk back end. I've already a prototype working after a few days of full immersion (I like to use vacation time to work at completely new ideas for Redis). The idea is that everything is stored on disk, in what is a plain key-value database (complex values are serialized when on disk), and the memory is instead used as an object cache. It is like taking current Redis Virtual Memory and inverting the logic completely, the result is the same (working set in memory, the rest on disk), but this implementation means that there are no limits on the data you can put into a single instance, that you don't have slow restarts (data is not loaded on memory if not demanded), and there isn't to fork() to save. Keys marked as "dirty" (modified) are transfered to disk asynchronously as needed, by IO threads. If everything will work as I expect (and initial tests are really encouraging) this means that Redis 2.4 will exit in a few months completely killing the current Virtual Memory implementation in favor of the new "two back ends" design, where you can select if you want to run an in-memory DB or an on-disk DB where memory is just an LRU cache for the working set.

kkovacs 16y ago
antirez, it's a honor that you commented, thanks! :) The new inverted logic for the VM you describe seems very interesting; I'm very much looking forward to see 2.4! Redis is already more than perfect what we use it for -- keeping track of stock price data, and distributing it. The size of the DB is known in advance (the amount of stocks does not grow very fast), and the performance is perfect. Keep up the good job! (And have a nice new year) Kristof
- antirez 16y ago

kkovacs 16y ago

I think it's a nice closing word from @jzy: A SQL query goes into a bar, walks up to two tables and asks, "Can I join you?" "No, but you can enjoy the view." Sorry :) K.

nl 16y ago

CouchDB & MongoDB both share one property that this comparison misses (or mentions only in passing). Both are schema free datastores. For me, this is the biggest, most useful difference between them and traditional SQL databases, because it makes things easy that are very, very hard (or inefficient) on an SQL database. It's probably also worth noting that other NoSQL solutions don't share this advantage. For example, Cassandra requires all nodes to be restarted to apply a schema change, which can be quite a big deal.

tylerhobbs 16y ago
"Cassandra requires all nodes to be restarted to apply a schema change, which can be quite a big deal." That's no longer true. In 0.7, keyspaces and column families may be created, altered, or dropped live.
- nl 16y ago
  I thought it might have been fixed by now. Anyway, you still need a schema with Cassandra.
  - rbranson 16y ago
    Not in the same sense as an SQL database. You can freely add columns and rows, just not Column Families or Keyspaces. This is because a KS+CF combo is stored in it's own file, in a certain order, so that it can be efficiently traversed using natural ordering. If you don't have this need and just need a flat K/V database, you can use a single KS+CF for everything.

markoa 16y ago

What we're missing are similar arricles that go into disadvantages and implications on deployment. Eg I have found out that deploying Tokyo Tyrant in a Rails project requires you to write some sčripts to ensure that things run properly. Also the db size has to be set in configuration in advance. MongoDB OTOH is not designed for a single server environment, has a very small max document size, easily gets corrupted if process is stopped etc.

dfhjtjty 16y ago

Click on our website: === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === "Priority, my friend!! Christmas is coming, quick to our website shopping, our web site shopping there will be something different, unexpected things to you, let you have different sense, our website wholesale various fashion shoes, such as Nike, Jordan, prada, also includes the jeans, shirt, bags, hats and decoration. All these products are our free transport, prices are competitive, we can also accept paypal j, after the payment within short time, can ship. = New era cap $12; Air jordan(1-24)shoes $30; jordan air max oakland raiders $34a€“39; Ed Hardy AF JUICY POLO Bikini $25; Christan Audigier BIKINI JACKET $25; Tshirts (Polo ,ed hardy,lacoste) $15 coogi DG edhardy gucci t-shirts $18; gstar coogi evisu true jeans $35; coach chanel gucci LV handbags $36; Sunglasses(Oakey,coach,gucci,A r m a i n i) $15; === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) === === ( http://www.etradinglife.com http://www.etradinglife.com ) ===

deleted 16y ago

[deleted]

redthrowaway 16y ago

So if Cassandra writes are much faster than reads, why would Reddit go that route? Their comment server is consistently breaking on them, and it would seem that a sub-optimal choice of db might be partly to blame.

cdavid 16y ago
Because it is normally harder to scale write than read. At least, that's the most obvious thing coming to my mind, I don't know reddit usage.
rbranson 16y ago
It's not as lop-sided as this article might have you believe and has largely been mitigated as of late. This is because Cassandra uses read repair, which is a big component of it's strategy to make both reads and writes to scale linearly while also ensuring durability. What is your suggestion otherwise? Any distributed database that is going to be inexpensive, performant, scalable, and durable will need to use some kind of quorum read repair system. Riak, Voldemort, and Dynamo all use read repair with high levels of production success.

sdgherhe 16y ago

-------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- Gold autumn, personality Men’s clothing Shoes, Travel bag that grabs an eye coat Chao packet Free transport -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 -------- -------- http://bit.ly/eTAvV5 http://bit.ly/eTAvV5 --------

benblack 16y ago

This article is mostly marketing phrases from the websites of the various projects. Sadly, much of it is inaccurate, extremely skewed, or otherwise not useful for the stated purpose of comparing the listed databases. For example, CouchDB having a "Main Point" of "DB consistency" might be the case, as it is for Redis, when there is no replication. In replicated configurations, it is definitely not true. Further, the MVCC is weaker in many ways than in a Dynamo system like Riak as you have no way to influence or discover consistency between replicas. I'm sure folks expert in other systems can identify similar errors in the rest of the post. Can someone explain to me who the target audience is for all these NoSQL comparison articles? They are universally poor, yet universally popular.

laran 16y ago

Nice writeup. A good survey of some key tools in the NoSQL space. Thanks!

deleted 16y ago

[deleted]

paxa 16y ago

Also is VertexDB - small graph database. It's written in C, uses Tokyo Cabinet for storing data. Simple http filesystem-like interface. The general advantage - links, that allow to make graph structures on database level. https://github.com/stevedekorte/vertexdb https://github.com/stevedekorte/vertexdb

Juha 16y ago

Thank's for the article, good information. Does anyone have any user amounts about the different no-sql databases? Or just say two most popular ones? I guess some of them will rise above the other's in following years and some will drop. User amounts would indicate which ones have most potential to stay around and be accepted as standard no-sql databases.

luca_garulli 16y ago

Good article, it's a good starting point to let the people to decide where to start in using a NoSQL solution. But what about OrientDB? Do you plan to add it in this feature comparison?