9 ms·
CouchDB 2.0
- k__ 10y agoTL;DR - Clustering https://blog.couchdb.org/2016/08/01/couchdb-2-0-architecture/ https://blog.couchdb.org/2016/08/01/couchdb-2-0-architecture... - New Query Language https://blog.couchdb.org/2016/08/03/feature-mango-query/ https://blog.couchdb.org/2016/08/03/feature-mango-query/ - New Admin Interface (written in React) https://blog.couchdb.org/2016/07/27/fauxton-the-new-couchdb-dashboard/ https://blog.couchdb.org/2016/07/27/fauxton-the-new-couchdb-...
- cbHXBY1D 10y agoAlso, 2.0 is the unification of BigCouch (Cloudant's work) with the old single node CouchDB.
- SEJeff 10y agoSad to not see Jepsen tests ran on this at least by the developers when releasing a new clustering piece. The folks working on cockroachdb recently did this[1] and it was a good read. [1] https://www.cockroachlabs.com/blog/diy-jepsen-testing-cockroachdb/ https://www.cockroachlabs.com/blog/diy-jepsen-testing-cockro...
- cbHXBY1D 10y agoCloudant did it for their clustering implementation of CouchDB. See here: https://cloudant.com/blog/run-dmc-explains-network-partitions/ https://cloudant.com/blog/run-dmc-explains-network-partition...
- robinson_k 10y agoCouchDB is eventually consistent and does not guarantee consistency. Would a Jepsen test make sense?
- spotman 10y agoOn their site they do say however: "And we care a lot about your data. CouchDB has a fault-tolerant storage engine that puts the safety of your data first." So, yeah, I think even if eventually consistent, and not immediately consistent, they want you to feel they have taken care of your data when designing the thing, so seems prudent to do.
- haimez 10y agoAs a rule, yes. I'm not at all an expert on (or even all that familiar with) CouchDB, but if you make any guarantees at all about your distributed storage engine, then those guarantees are somehow testable.
- SEJeff 10y agoYes. Eventual consistent doesn't mean inconsistent, but eventually it will be. That means under a network partition (P of th CAP Theorem) things should converge after the network converges. This is precisely the type of semantic that Aphyr wrote Jepsen for. See his testing of Cassandra (eventually consistent) or the CockroachDB team (eventually consistent) or Risk (eventually consistent) running Jespsen to prove things work as expected.
- SEJeff 10y agoAnd by Risk I meant Riak but didn't notice it in time to edit the comment (sorry).
- janl 10y agoWe’d definitely love a Jepsen write up, preferably by @aphyr himself. We just need to find out how to get that going :)
- SEJeff 10y agoHere you go! Kyle is available to hire to do Jepsen testing if you're serious about it. Here is the page on how to go about it: http://jepsen.io/analyses.html http://jepsen.io/analyses.html
- bdcravens 10y ago> The second major feature is the declarative query language “Mango”. Nice name :-)
- Direct 10y agoI wonder if the design is based on some of the solid research that went into MangoDB[0]. [0]: https://github.com/dcramer/mangodb https://github.com/dcramer/mangodb
- SwellJoe 10y agoMango is also the name of a popular MongoDB driver for Perl, most commonly used with Mojolicious. I'm not sure which came first. http://search.cpan.org/~odc/Mango-1.29/lib/Mango.pm http://search.cpan.org/~odc/Mango-1.29/lib/Mango.pm
- janl 10y agoIt’s MongoDB inspired obviously, but at the time MongoDB asked IBM/Cloudant to not call it that. They settled on Mango. The Cloudant product then became “Cloudant Query”, so CouchDB easily inherited the Mango nickname. We like it :)
- eriknstr 10y agoI bought the first edition of an O'Reilly book called CouchDB: The Definitive Guide a few years back. Looking at the online draft version of the book, the tour chapter [0] still has version 0.10.1 in the example. I wonder if there will be a second edition of the book covering mango, clustering and the new admin interface. [0]: http://guide.couchdb.org/draft/tour.html http://guide.couchdb.org/draft/tour.html
- fiatjaf 10y agoYou don't need a book for learning an admin interface.
- eriknstr 10y agoI don't "need" anything in a book, but it sure was handy to have a full, detailed breakdown with a lot of pictures when I read the first edition of the book without first having to installing some software (CouchDB) that, at the time when I bought the book to learn about it, I did not know if I had actual use for, and did not want to spend time finding my way through its interface in order to find out. The admin interface of 2.0 will probably be familiar since I've already learned my way around older releases. Nonetheless, I think that there is no question about the fact that if there comes a second edition of the book that is going to cover 2.0, and seeing as how the first edition has some stuff about the admin interface, that's obviously going to need to be updated, no question about it. So when I buy the next edition of the book, I will probably just skim through the admin interface stuff, but not all readers will have used a previous version of CouchDB, so to some of them, having that covered and up to date will be valuable like it was to me the first time I wanted to know about CouchDB. So if you think I'm saying that I need the book to learn the admin interface, I think you did not understand what I meant, because that is not what I meant. I meant that I want to learn about mango and clustering, and that I also expect the next edition of the book to have the parts about the admin interface updated.
- fiatjaf 10y agoOk, I don't think you meant you need the book just for the admin interface, I just made a jerky comment. I was, however, under the impression that you actually needed a book for a lot of things and I didn't like that feeling. However, now that you explained, I understand how good can a good book be. I also read that CouchDB book almost entirely before going on to use the database, before thinking about it, or having a need for it, and that made wonders to me. I had forgotten about this fact of my life. Thank you for reminding me.
- matt4077 10y agoYet another product blog that doesn't manage to prominently link to the product. If you want to make my life a tiny bit easier: - blog.<project>.org needs a link to <project>.org because half of you visitors want to read the "it" before they read the "news". - github.com/<you>/<Transmogrifier>For<ThatProject> should have a link to <ThatProject> in the description or the readme's first paragraph. I often come across interesting plugins without knowing anything about projects' they are for. (and yes, I know there are actual problems in the world. But these are easy to solve)
- happytrails 10y agoI agree, but I imagine your blood pressure is through the roof with such a critique.
- shash7 10y agoSo true. Half of the time when I find myself on a engineering blog for a startup, I am curious about their product but there is no link to their main page. Clicking on the logo redirects me to blog.<startup>.com
- jonursenbach 10y agoThere's an "About" link on the blog that literally explains what it is.
- janl 10y agoExcept, you know, for the “Download” headline that I specifically put in for you ;)
- SkyMarshal 10y agoWhy is this the top post
- deleted 10y ago[deleted]
- eriknstr 10y agoJust installed it on FreeBSD 10.3. Fetched and unpacked the source tarball, then did mostly what the INSTALL.Unix.md said to do; sudo pkg install erlang icu spidermonkey185 gmake gcc curl help2man py27-sphinx ./configure gmake release sudo pw useradd couchdb -u 5984 -c "CouchDB Administrator" -L daemon -s /usr/local/bin/bash sudo cp -R rel/couchdb /home/couchdb sudo chown -R couchdb:couchdb /home/couchdb sudo find /home/couchdb -type d -exec chmod 0770 {} \; sudo find /home/couchdb/etc/ -type f -exec chmod 0644 {} \; Note how I set the uid to 5984 ;) Unfortunately, when I try to actually run it... sudo -i -u couchdb ~couchdb/bin/couchdb ...I'm just getting messages like: [error] 2016-09-20T23:57:55.321596Z couchdb@localhost emulator -------- Error in process <0.547.0> on node couchdb@localhost with exit value: {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users", [{file,"src/mem3_shards.erl"},{line,327}]},{mem3_shards,load_shards_from_disk,1, [{file,"src/mem3_shards.erl"},{line,315}]},{mem3_shards,load_shards_from_disk,2, [{file,"src/mem3_shards.erl"},{line,331}]},{mem3_shards,for_docid,3, [{file,"src/mem3_shards.erl"},{line,87}]},{fabric_doc_open,go,3, [{file,"src/fabric_doc_open.erl"},{line,38}]}, {chttpd_auth_cache,ensure_auth_ddoc_exists,2, [{file,"src/chttpd_auth_cache.erl"},{line,187}]}, {chttpd_auth_cache,listen_for_changes,1, [{file,"src/chttpd_auth_cache.erl"},{line,134}]}]} [notice] 2016-09-20T23:57:55.321650Z couchdb@localhost <0.323.0> -------- chttpd_auth_cache changes listener died database_does_not_exist at mem3_shards:load_shards_from_db/6(line:327) <= mem3_shards:load_shards_from_disk/1(line:315) <= mem3_shards:load_shards_from_disk/2(line:331) <= mem3_shards:for_docid/3(line:87) <= fabric_doc_open:go/3(line:38) <= chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:187) <= chttpd_auth_cache:listen_for_changes/1(line:134)
- chrisfosterelli 10y agoThe new clustered instance doesn't autocreate the system tables. It used to do this in 1.x, but no longer does. You'll have to create these tables: * _global_changes * _metadata * _replicator * _users It's a bit of a pain!
- janl 10y agoThe admin interface has a convenient “Setup” menu point that does all this for you :)
- desireco42 10y agoI remember CouchDB when it was initially released, it was one of the most promising and innovative DB's around. It's a shame it didn't live up to the hype.
- ohstopitu 10y agowhy do you feel that it didn't? (genuinely curious) I used it for a variety of projects and seem to love it.
- qwertyuiop924 10y agoI actually quite like Couch. It's an interesting system, and I think it doesn't get enough credit.
- rdtsc 10y ago> , it was one of the most promising and innovative DB's around. It's a shame it didn't live up to the hype. It promised master-to-master replication, so has that. Doesn't lose your data (actually fsyncs, yay!). Has a helpful integrated web interface, is HTTP + JSON interface. I don't know I've shipped lots of products on it. I say it lived up pretty well to the hype...
- fet 10y agoHype or not, I inherited a project that had CouchDB baked into the bones. While I found its disk space usage and its time to reindex large databases frustrating, I did appreciate REST-like queries, how it just saved whatever you wanted and how the map/reduce queries could actually be quite powerful for ETL and reporting (though it took a good while to index every time we had a change). We abandoned replication early on as just too chatty over high latency networks. I love the CouchApp idea because it's a natural extension of "save whatever you want." As long as we're saving json, why not save static files too? It lets you prototype quickly and, in our case, let us develop and test out specific features for clients by quarantining risky crap code on the fly into one database. I like to think we're beyond that stage, but at the time it was invaluable to keep multiple versions of our app running concurrently. I have 3 solid years of experience working with CouchDB and in the end I find myself longing for PostgreSQL with one bson column for the unknown or "volatile" attributes. Our data was semi-relational, as I would argue is most data. Meaning there were honest to goodness has-many or belongs-to type queries that could have been simplified and easier to maintain outside the application code. I know that's just a style, some people would use key-value stores for everything with no validation on anything. If you're not that far gone and you still like freedom then CouchDB might be for you. As for me, I wanted an error from my database if my application code asked for something or tried to store something invalid. But for rapid prototyping for us I don't think we could have gotten a better database.
- drhayes9 10y agoI love Couch and I feel totally relaxed. What are people building with it? A spiffy log package was the first thing I thought of and indeed that's one idea listed on this page: http://docs.couchdb.org/en/2.0.0/intro/why.html http://docs.couchdb.org/en/2.0.0/intro/why.html But I think the "logging" case is hard to argue vs. "manually" grepping (or ag-ing using the silver searcher) over in-place log files and aggregating/rendering dashboards via static files.
- WorldMaker 10y agoNot using Couch directly at the moment, but have several projects I'm working on that are using Pouch as an easy to sync, offline-first data store. (These are largely targeting Cordova/Electron.)
- teddyc 10y agoI used it to store user-uploaded images in a photo contest website years ago. It was a happy medium between storing images directly on the filesystem or as a blob/binary in a relational database. The image requests were AJAX, so I could include the clients screen dimension in the request. My app would resize on the fly if necessary, store resized image to CouchDB, and redirect the image request to CouchDB, which was publicly readable. Another good use case I had was for JSONP request for an auto-complete input field on a webpage. Again, the database was publicly readable. I have also used it to aggregate data for graphs. The data changed daily, so the ability to cache the results of a view until the data changed was nice. But the first request of data each day still took a while. I don't think I got a performance boost in this case, but I did get free caching. All my other uses of CouchDB were mostly for fun and could've been implemented in traditional SQL.
- fiatjaf 10y agoAs an low-profile developer who's been using CouchDB for a long time, some weeks ago I've written some quick personal opinions on what CouchDB has become: http://fiatjaf.on.flowi.es/about-couchdb/ http://fiatjaf.on.flowi.es/about-couchdb/
- drhayes9 10y agoI really like your write-up. I'm sad that the Couchapp thing never took off or is getting de-emphasized. That was one of the really brain-bendy ideas about Couch that I loved, that these DB-side applications could also replicate to other people. I'm only just catching up with Couch and finding the db-per-user stuff. Do you know of anyone shipping a Couch instance inside thick clients, like a traditional desktop application?
- brightball 10y agoPouchDB is my best guess
- davidbanham 10y agoAgreed. PouchDB is the go to if you want an "embedded couch".
- fiatjaf 10y agoI guess CouchDB is not the easiest thing to install in a non-technical person computer, but PouchDB, on the other hand, is as easy as it can get. I don't understand how projects like remotestorage, for example, aren't using Pouch and Couch and instead try to develop their own replication protocol without success for many years.
- pokstad 10y agoMy thoughts exactly. I'm not sure who CouchDB is supposed to be for anymore. The original developers who fell in love with it are moving on. Who is the audience for 2.0? Who does IBM hope to reach with Cloudant? BTW, you are spot on about continuous replication. Try using continuous replication on Cloudant and you will end up with a fat bill.
- qwertyuiop924 10y agoI'm actually really excited for this. There seem to be a lot of new, interesting, and useful features, and I'm already a huge fan of the couch model.
- hiphipjorge 10y agoThe UI looks pretty awesome. Kind of see the RethinkDB influence in shipping with a web admin, and it's great that where things are heading.
- chromakode 10y agoCouch did it first!
- marknadal 10y agoCongrats to the CouchDB team! I remember playing around with writing a NodeJS driver for it all the way back in 2011. Sadly, I decided to use and make a driver for MongoDB instead simply because it was easier at the time. Despite that choice, I have always admired CouchDB for pushing the frontiers on stuff like Offline-First with PouchDB way ahead of its time (and also what inspired me a little bit for our Open Source Firebase alternative http://gun.js.org/ http://gun.js.org/ ). I am particularly really excited to see the announcement of the Mango query language, since querying Couch was one of lesser-easy things to do back then. I'm also very excited to hear about performance improvements, as this has been particularly interesting to me as I've been tracking various system's performance as I have worked on our own (Mongo with Wired Tiger, Cassandra, Redis, and even Chrome V8 engine as we have built towards 30M+ ops/sec, see https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2GB-Atom-CPU https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2... ). However clicking through on the performance links didn't lead to any numbers or benchmarks. I would love to see that! Really happy that Couch is on the homepage of Hacker News. I really feel like they made lots of correct decisions that got passed over by the NoSQL craze, and have lately not been receiving the type of attention as it should compared to (what I biasly think) unfavorable but hyped up Master-Slave systems. People should really check into Couch's Master-Master replication!
- m_mueller 10y agoSo, as a CouchDB 1.x user, where to go next? CouchDB 2.x or Couchbase?
- rdtsc 10y agoCouchDB 2.0 is a direct descendant of CouchDB 1.x. The API is 99% compatible, same community working on it, you can replicate between them and so on. I haven't used Couchbase, but I understand besides the "Couch" prefix and that CouchDB's original author working there for a few years, it doesn't have much in common with CouchDB project.
- m_mueller 10y agoWell Couchbase Lite is largely compatible as far as I can attest. Not sure how it is with Couchbase proper, but I was under the assumption that it is API compatible to CBL.
- janl 10y agoCouchbase has a different API, so it is not an “upgrade”, more a rewrite. CouchDB 2 is 99% API compatible with version 1.
- cbsmith 10y agoOn the plus side, Couchbase does support the memcache protocol and has a lot of nice features that CouchDB doesn't have.
- maxpert 10y agoGood to see v2.0 took them quite long but hey better late then never.
- nowprovision 10y agoA fews things I wanted out of CouchDB years back: - faster bulk indexing - space reduction, I think a simple couchdb to psql json was 1/10th of size - ES6 or even ES5 - * its been a few years since I last looked but I remember you had to tread carefully - Object.keys maybe was one? When I started with CouchDB it wrong choice for so many reasons, client had <30gb of data, couchdb was cooler than node.js, and I was frustrated with SQL Server. In hindsight sticking with SQL Server or Postgresql would of been better - older/wiser today.
- reitoei 10y agoClassic NoSQL adopter bantz.
- janl 10y ago> - faster bulk indexing With clustering you now get that. By way of “oversharding” even on a single node. Speed up is linear with number of shards / CPUs > - space reduction, 2.0 has the better compaction format. There are still ways to improve, but we are getting there. > - ES6 or even ES5 Our custom wrapper around Spidermonkey 185 is getting long in the tooth. It didn’t make it into 2.0, but we are well aware that this needs updating. Anyway, sounds like we got there in the end ;)
- mehh 10y agoSo its taken about 3 years to get from the alpha to release .. meh! Whilst I really like couch I'm just not convinced on using it in production due to its glacial pace.
- wildchild 10y agoUseless abandonware.
- Kristine1975 10y agoThe first thing I think upon hearing CouchDB is "perform like a pr0n star"...