5 ms·

I find it unfortunate that CouchDB is pretty much a dead project even though it's fundamentals are for most projects better than MongoDB. Master-master database

by nawitus 12y ago

I find it unfortunate that CouchDB is pretty much a dead project even though it's fundamentals are for most projects better than MongoDB. Master-master databases are the future for most scaled architectures.

ddorian43 12y ago
The indexing interface sucked. I remember it's storage growed too fast(mongo also). Not webscale enough like mongodb.
tracker1 12y ago
MongoDB does master-slave replication with an election protocol to automagically promote a slave on failure. The use of this in practice is pretty good. It really depends on what you need... there are situations where I would recommend MongoDB, RethinkDB, ElasticSearch or Cassandra... The farther you get from traditional RDBMS, the more you have to consider your needs and any trade offs.
- nawitus 12y ago
  What I find interesting is that every single time I bring up MongoDB's lack of master-master architecture/replication on Hacker News the response I get is related to the fact that MongoDB has a master-slave architecture. Sure, it might work for some applications well, but master-master is a better choice for typical NoSQL uses. The whole point of master-master is not dealing with failure situations or trying to scale out performance (even though master-master handles those situations naturally), the point is that it's a different philosophy. The idea is that there's no single official true state of the database. Reality happens to map that idea very well. The data doesn't exist in a centralized official place, it exists in multiple places which may not be perfectly connected in real time all the time. The data exists at multiple servers at multiple databases. It exists on your mobile device, sometimes disconnected from the server. It exists on thousands of browsers on the same time. There's no single master. The best mapping of this reality is the master-master ideology. Yes, it requires application-level merges of data, but the benefits you get from implementing this can be tremendous. And that's why I'm excited about using CouchDB. Or rather I'm excited about master-master databases. CouchDB happens to be one of those, and I hope future databases will be built around that too.
- TheAceOfHearts 12y ago
  Can you expand upon that? When would you recommend MongoDB, RethinkDB, ElasticSearch, or Cassandra?
  - tracker1 12y ago
    If you need massive reads or writes... I'm talking hundreds of thousands of simultaneous users, then Cassandra is probably your best bet, the tooling around your use will be a bit more difficult, you may need to shard/replicate portions of your data anyway depending on use. You reach for Cassandra when your use is going to get complicated. I wouldn't look at Cassandra if I needed less than say a dozen nodes to start with. ElasticSearch is fairly close, but I'm unsure if it will scale as well in practice. ElasticSearch handles the middle-to-high ground very well. I'm not sure I would use it for authorative data... It works incredibly well for logging (with logstash) and the front end utilities (kibana, etc) are nice too. It's primary use is as a search, if you are very read heavy for searches, or write heavy for analytical data it works well. You can tune your use to separate the storage and reads in interesting ways. MongoDB works incredibly well when your core data is composed of mostly self-contained documents and in need of certain flexibility. A typical classifieds website is a great use case. It's also a very natural fit in a lot of programming languages, js (node) in particular is a natural fit. There's much less disconnect between the data and your application models. RethinkDB is very similar to MongoDB, but has a more traditional mindset when it comes to it's use. I think the programming interfaces are a little better thought out, consistency and data security is at the forefront here. In general it comes down to... very large loads, use cassandra... easiest use is mongo... rethinkdb I've been waiting on the replication story to get better, and the geosearch support is fairly recent... ElasticSearch search or for logging (write heavy). In many scenarios where I would use ElasticSearch, it would be along with an RDBMS as an authoratative data source.
- willholley 12y ago
  That's fine for scaling reads - writes, not so much. The leader election in MongoDB has been problematic in the past (https://aphyr.com/posts/284-call-me-maybe-mongodb https://aphyr.com/posts/284-call-me-maybe-mongodb) but I think they have made recent moves to address this.
Lazare 12y ago
While the project is, if not dead, then certainly a bit quiet, I think the protocol is still very much alive. Some very interesting stuff happening with PouchDB these days. And it's the protocol that enables the sexy master-master replication, so...not all bad news.
jgalt212 12y ago
I hear what you are saying about the value of master-master, but querying on CouchDB is rather arduous, while querying on MongoDB (if you are coming from SQL world) is quite easy. That being said, CouchDB does have a nice built-in in gui.
- cmpb 12y ago
  Genghisapp is a great drop-in GUI for Mongo btw.
- conorgil145 12y ago
  my biggest complaints about CouchDB revolve around design documents. The documentation on which portion of a design document will trigger the indices to recalculate are awful. I had to figure it out by asking in IRC and then still doing trial/error. Cloudant is doing some very interesting things relating to querying on a CoudhDB system. They created a query syntax so that you have an option besides map/reduce [1]. I watched their webinar on it and it seems pretty slick, but I have not yet played with it. Also, it is not part of CouchDB yet so there is no option to run tests against locally to verify syntax, response errors, etc, etc. Cloudant is working to merge many of their changes from BigCouch back into the CouchDB project, so one day I expect CouchDB to have multiple query options [2]. [1] https://cloudant.com/blog/introducing-cloudant-query/#.VLCSc83d_dE https://cloudant.com/blog/introducing-cloudant-query/#.VLCSc... [2] https://cloudant.com/blog/update-from-nebraska-the-cloudant-couchdb-merger/#.VLCS483d_dE https://cloudant.com/blog/update-from-nebraska-the-cloudant-...
  - brickcap 12y ago
    Yeah it was a bit confusing for me too initially but if anyone else still does not know the answer to this it is:- 1. Indexes are built and appended to at read time. 2. If you change the view code an index rebuilding will occur. (A hash of all the views in the design document is taken and compared with the previous hash. So even a small thing such as adding a space will in effect change the view code and trigger rebuilding of views.) 3. No change to any other part of design document will have any effect on views. I wrote in detail about it here (http://staticshin.com/programming/does-updating-a-design-document-in-couchdb-cause-rebuilding-of-views/ http://staticshin.com/programming/does-updating-a-design-doc...)
otterley 12y ago
What can you do in CouchDB that you can't do in ElasticSearch?
- vvoyer 12y ago
  localDB.replicate.to(remoteDB); maybe http://pouchdb.com/guides/replication.html http://pouchdb.com/guides/replication.html
- brickcap 12y ago
  @vvoyer already mentioned replication. But there is another thing that couchdb is good at but most people don't use it/ know about it. It can interpret arbitrary erlang code. Which means that you can write an erlang module and call it with couchdb show functions and have the benefit of a ready made http api for you. A practical use of this would be to use mnesia as an in memory key-value store(for similar things that you would use redis for) and then call the functions in the module from a show function. All the benefits that you get with erlang you get with couchdb. One of the killer features (for me at least) is user accounts management. In 3 http calls I have login-logout facility in my app ready. I can't tell you how impressed some of my clients are when I show them a V1 of their product in a couple of days.
  - otterley 12y ago
    If you configure your indexes properly, you get replication in ES for free: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/replica-shards.html#replica-shards http://www.elasticsearch.org/guide/en/elasticsearch/guide/cu...
    - janl 12y ago
      This is different from CouchDB’s peer-to-peer or master-less replication where nodes can live all over the world, be offline for arbitrary amounts of time or even be phones or web-browsers.
- willholley 12y ago
  ES isn't really meant to be used as an operational datastore (although I appreciate some do use it that way). CouchDB provides strong durability and master-master replication. It's also very easy to sync changes from CouchDB (https://github.com/elasticsearch/elasticsearch-river-couchdb https://github.com/elasticsearch/elasticsearch-river-couchdb) to ES so they make a good complimentary pair. Use CouchDB for the raw data (where you want strong durability and write scalability) and use ES to search over it.
logn 12y ago
Cloudant developed BigCouch which is an enhanced CouchDB for clusters. Last I heard they intended to merge their fork back in for the next major rev. IBM bought Cloudant and in my experience, IBM does a pretty good job with open source (e.g., Eclipse).
ahoge 12y ago
CouchDB just isn't very nice to use. Compared to MongoDB or RethinkDB, it really isn't any fun. You can't do ad-hoc queries and it doesn't give you the tools to properly write/test your map/reduce/rereduce functions. I still don't know how trying the rereduce step is supposed to work. To make matters worse, the used JS engine is fairly old and the exceedingly wordy documentation is kinda hard to follow. I'm a lot happier with RethinkDB.
- brickcap 12y ago
  Agreed about ad-hoc queries. It would have been nice to have them but couchdb-lucene[1] or even an integration with elasticsearch helps a lot if you need them. https://github.com/rnewson/couchdb-lucene https://github.com/rnewson/couchdb-lucene
willholley 12y ago
Although it's been a quiet few years, the project is on the up again. 2014 was a big year for CouchDB - CouchDB 2.0 developer preview was released (more on this in a sec) [1], IBM acquired Cloudant (and has made it available as on-premise offering), Cloudant Query (MongoDB-style querying for CouchDB)[2] was contributed to CouchDB. PouchDB, Couchbase Mobile and Cloudant Sync (all replication-compatible with CouchDB) all saw major new releases and increased uptake. Much more detail on all of this is on the CouchDB blog [3]. CouchDB 2.0 adds dynamo-style clustering support (similar to Riak and Cassandra) in addition to the replication(sync) protocol used by the CouchDB ecosystem [4]. It also includes all the fixes / performance improvements based on Cloudant's operational experience over the last 5 years. That said, document databases are certainly a niche. CouchDB is a good choice if you need strong durability (writes are always fsync'd - to multiple copies when clustering) and consistent performance as your database scales (querying options may seem restrictive but are designed to scale very well). As others have pointed out, the RESTful interface to CouchDB makes it a good fit for web and mobile applications which can query the database directly without an app tier. PouchDB [5] and various mobile datastores which implement the sync protocol [6][7] allow you to also take your database to a browser or mobile client and work with it when disconnected which is pretty compelling. [1] https://speakerdeck.com/wohali/putting-the-c-back-in-couchdb-plus-query https://speakerdeck.com/wohali/putting-the-c-back-in-couchdb... [2] https://cloudant.com/blog/introducing-cloudant-query https://cloudant.com/blog/introducing-cloudant-query [3] http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-december-19-2014-and-couchdb-highlights-in-2014/ http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-decem... [4] http://www.replication.io/ http://www.replication.io/ [5] http://pouchdb.com/ http://pouchdb.com/ [6] https://cloudant.com/cloudant-sync-resources https://cloudant.com/cloudant-sync-resources [7] http://www.couchbase.com/nosql-databases/couchbase-mobile http://www.couchbase.com/nosql-databases/couchbase-mobile
ensby 12y ago
Hi nawitus, how do you define "pretty much a dead project". I have been looking at the decline in google trend and, yes, it looks like it is dying, but the activity in the group is very much alive. And IBM Cloudant is now backing it with very nice integration to the tools of the corporate world.

by nawitus 12y ago

ddorian43 12y ago

The indexing interface sucked. I remember it's storage growed too fast(mongo also). Not webscale enough like mongodb.

tracker1 12y ago

MongoDB does master-slave replication with an election protocol to automagically promote a slave on failure. The use of this in practice is pretty good. It really depends on what you need... there are situations where I would recommend MongoDB, RethinkDB, ElasticSearch or Cassandra... The farther you get from traditional RDBMS, the more you have to consider your needs and any trade offs.

nawitus 12y ago
What I find interesting is that every single time I bring up MongoDB's lack of master-master architecture/replication on Hacker News the response I get is related to the fact that MongoDB has a master-slave architecture. Sure, it might work for some applications well, but master-master is a better choice for typical NoSQL uses. The whole point of master-master is not dealing with failure situations or trying to scale out performance (even though master-master handles those situations naturally), the point is that it's a different philosophy. The idea is that there's no single official true state of the database. Reality happens to map that idea very well. The data doesn't exist in a centralized official place, it exists in multiple places which may not be perfectly connected in real time all the time. The data exists at multiple servers at multiple databases. It exists on your mobile device, sometimes disconnected from the server. It exists on thousands of browsers on the same time. There's no single master. The best mapping of this reality is the master-master ideology. Yes, it requires application-level merges of data, but the benefits you get from implementing this can be tremendous. And that's why I'm excited about using CouchDB. Or rather I'm excited about master-master databases. CouchDB happens to be one of those, and I hope future databases will be built around that too.
TheAceOfHearts 12y ago
Can you expand upon that? When would you recommend MongoDB, RethinkDB, ElasticSearch, or Cassandra?
- tracker1 12y ago
  If you need massive reads or writes... I'm talking hundreds of thousands of simultaneous users, then Cassandra is probably your best bet, the tooling around your use will be a bit more difficult, you may need to shard/replicate portions of your data anyway depending on use. You reach for Cassandra when your use is going to get complicated. I wouldn't look at Cassandra if I needed less than say a dozen nodes to start with. ElasticSearch is fairly close, but I'm unsure if it will scale as well in practice. ElasticSearch handles the middle-to-high ground very well. I'm not sure I would use it for authorative data... It works incredibly well for logging (with logstash) and the front end utilities (kibana, etc) are nice too. It's primary use is as a search, if you are very read heavy for searches, or write heavy for analytical data it works well. You can tune your use to separate the storage and reads in interesting ways. MongoDB works incredibly well when your core data is composed of mostly self-contained documents and in need of certain flexibility. A typical classifieds website is a great use case. It's also a very natural fit in a lot of programming languages, js (node) in particular is a natural fit. There's much less disconnect between the data and your application models. RethinkDB is very similar to MongoDB, but has a more traditional mindset when it comes to it's use. I think the programming interfaces are a little better thought out, consistency and data security is at the forefront here. In general it comes down to... very large loads, use cassandra... easiest use is mongo... rethinkdb I've been waiting on the replication story to get better, and the geosearch support is fairly recent... ElasticSearch search or for logging (write heavy). In many scenarios where I would use ElasticSearch, it would be along with an RDBMS as an authoratative data source.
willholley 12y ago
That's fine for scaling reads - writes, not so much. The leader election in MongoDB has been problematic in the past (https://aphyr.com/posts/284-call-me-maybe-mongodb https://aphyr.com/posts/284-call-me-maybe-mongodb) but I think they have made recent moves to address this.

Lazare 12y ago

While the project is, if not dead, then certainly a bit quiet, I think the protocol is still very much alive. Some very interesting stuff happening with PouchDB these days. And it's the protocol that enables the sexy master-master replication, so...not all bad news.

jgalt212 12y ago

I hear what you are saying about the value of master-master, but querying on CouchDB is rather arduous, while querying on MongoDB (if you are coming from SQL world) is quite easy. That being said, CouchDB does have a nice built-in in gui.

cmpb 12y ago
Genghisapp is a great drop-in GUI for Mongo btw.
conorgil145 12y ago
my biggest complaints about CouchDB revolve around design documents. The documentation on which portion of a design document will trigger the indices to recalculate are awful. I had to figure it out by asking in IRC and then still doing trial/error. Cloudant is doing some very interesting things relating to querying on a CoudhDB system. They created a query syntax so that you have an option besides map/reduce [1]. I watched their webinar on it and it seems pretty slick, but I have not yet played with it. Also, it is not part of CouchDB yet so there is no option to run tests against locally to verify syntax, response errors, etc, etc. Cloudant is working to merge many of their changes from BigCouch back into the CouchDB project, so one day I expect CouchDB to have multiple query options [2]. [1] https://cloudant.com/blog/introducing-cloudant-query/#.VLCSc83d_dE https://cloudant.com/blog/introducing-cloudant-query/#.VLCSc... [2] https://cloudant.com/blog/update-from-nebraska-the-cloudant-couchdb-merger/#.VLCS483d_dE https://cloudant.com/blog/update-from-nebraska-the-cloudant-...
- brickcap 12y ago
  Yeah it was a bit confusing for me too initially but if anyone else still does not know the answer to this it is:- 1. Indexes are built and appended to at read time. 2. If you change the view code an index rebuilding will occur. (A hash of all the views in the design document is taken and compared with the previous hash. So even a small thing such as adding a space will in effect change the view code and trigger rebuilding of views.) 3. No change to any other part of design document will have any effect on views. I wrote in detail about it here (http://staticshin.com/programming/does-updating-a-design-document-in-couchdb-cause-rebuilding-of-views/ http://staticshin.com/programming/does-updating-a-design-doc...)

otterley 12y ago

What can you do in CouchDB that you can't do in ElasticSearch?

vvoyer 12y ago
localDB.replicate.to(remoteDB); maybe http://pouchdb.com/guides/replication.html http://pouchdb.com/guides/replication.html
brickcap 12y ago
@vvoyer already mentioned replication. But there is another thing that couchdb is good at but most people don't use it/ know about it. It can interpret arbitrary erlang code. Which means that you can write an erlang module and call it with couchdb show functions and have the benefit of a ready made http api for you. A practical use of this would be to use mnesia as an in memory key-value store(for similar things that you would use redis for) and then call the functions in the module from a show function. All the benefits that you get with erlang you get with couchdb. One of the killer features (for me at least) is user accounts management. In 3 http calls I have login-logout facility in my app ready. I can't tell you how impressed some of my clients are when I show them a V1 of their product in a couple of days.
- otterley 12y ago
  If you configure your indexes properly, you get replication in ES for free: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/replica-shards.html#replica-shards http://www.elasticsearch.org/guide/en/elasticsearch/guide/cu...
  - janl 12y ago
    This is different from CouchDB’s peer-to-peer or master-less replication where nodes can live all over the world, be offline for arbitrary amounts of time or even be phones or web-browsers.
willholley 12y ago
ES isn't really meant to be used as an operational datastore (although I appreciate some do use it that way). CouchDB provides strong durability and master-master replication. It's also very easy to sync changes from CouchDB (https://github.com/elasticsearch/elasticsearch-river-couchdb https://github.com/elasticsearch/elasticsearch-river-couchdb) to ES so they make a good complimentary pair. Use CouchDB for the raw data (where you want strong durability and write scalability) and use ES to search over it.

logn 12y ago

Cloudant developed BigCouch which is an enhanced CouchDB for clusters. Last I heard they intended to merge their fork back in for the next major rev. IBM bought Cloudant and in my experience, IBM does a pretty good job with open source (e.g., Eclipse).

ahoge 12y ago

CouchDB just isn't very nice to use. Compared to MongoDB or RethinkDB, it really isn't any fun. You can't do ad-hoc queries and it doesn't give you the tools to properly write/test your map/reduce/rereduce functions. I still don't know how trying the rereduce step is supposed to work. To make matters worse, the used JS engine is fairly old and the exceedingly wordy documentation is kinda hard to follow. I'm a lot happier with RethinkDB.

brickcap 12y ago
Agreed about ad-hoc queries. It would have been nice to have them but couchdb-lucene[1] or even an integration with elasticsearch helps a lot if you need them. https://github.com/rnewson/couchdb-lucene https://github.com/rnewson/couchdb-lucene

willholley 12y ago

Although it's been a quiet few years, the project is on the up again. 2014 was a big year for CouchDB - CouchDB 2.0 developer preview was released (more on this in a sec) [1], IBM acquired Cloudant (and has made it available as on-premise offering), Cloudant Query (MongoDB-style querying for CouchDB)[2] was contributed to CouchDB. PouchDB, Couchbase Mobile and Cloudant Sync (all replication-compatible with CouchDB) all saw major new releases and increased uptake. Much more detail on all of this is on the CouchDB blog [3]. CouchDB 2.0 adds dynamo-style clustering support (similar to Riak and Cassandra) in addition to the replication(sync) protocol used by the CouchDB ecosystem [4]. It also includes all the fixes / performance improvements based on Cloudant's operational experience over the last 5 years. That said, document databases are certainly a niche. CouchDB is a good choice if you need strong durability (writes are always fsync'd - to multiple copies when clustering) and consistent performance as your database scales (querying options may seem restrictive but are designed to scale very well). As others have pointed out, the RESTful interface to CouchDB makes it a good fit for web and mobile applications which can query the database directly without an app tier. PouchDB [5] and various mobile datastores which implement the sync protocol [6][7] allow you to also take your database to a browser or mobile client and work with it when disconnected which is pretty compelling. [1] https://speakerdeck.com/wohali/putting-the-c-back-in-couchdb-plus-query https://speakerdeck.com/wohali/putting-the-c-back-in-couchdb... [2] https://cloudant.com/blog/introducing-cloudant-query https://cloudant.com/blog/introducing-cloudant-query [3] http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-december-19-2014-and-couchdb-highlights-in-2014/ http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-decem... [4] http://www.replication.io/ http://www.replication.io/ [5] http://pouchdb.com/ http://pouchdb.com/ [6] https://cloudant.com/cloudant-sync-resources https://cloudant.com/cloudant-sync-resources [7] http://www.couchbase.com/nosql-databases/couchbase-mobile http://www.couchbase.com/nosql-databases/couchbase-mobile

ensby 12y ago

Hi nawitus, how do you define "pretty much a dead project". I have been looking at the decline in google trend and, yes, it looks like it is dying, but the activity in the group is very much alive. And IBM Cloudant is now backing it with very nice integration to the tools of the corporate world.