4 ms·

Slightly off topic but the general hacker community seems to somehow missed it-- the creators of CouchDB formed a company and merged with the creators of Memcac

by MCRed 11y ago

Slightly off topic but the general hacker community seems to somehow missed it-- the creators of CouchDB formed a company and merged with the creators of Memcached, and the new company is called Couchbase. This is the best NoSQL database going. Memcached built in, CouchDB views, scalable (really actually scalable, not mongodb "scalable") etc. I've long thought the hacker culture ignored databases and just picked something because it was popular (eg mysql) even though there were superior (objectively superior- we are engineers after all) solutions out there. Erlang is one of those languages that is objectively superior -- I've yet to meet another language that does concurrency right-- yet many hackers just ignore it because it's not got java's syntax. Which is silly. Don't make the same mistake with next generation databases.

ddorian43 11y ago
*This is the best NoSQL database going. Yeah, right. Most of the things about it ARE wrong: 1. couchdb views, who likes async map-reduce indexes ? 2. memcached built ok (better would be redis built in) 3. json documents, even mongodb has bson and not json 4. the new global-indexes-thing IS NOT scalable because you have to hit every index-node to do a query 5. when will you be able to modify a document ? looks like still in beta
- dang 11y ago
  > Yeah, right. Most of the things about it ARE wrong You've clearly got some good points here, but please don't break the HN guidelines by introducing them that way.
  - ddorian43 11y ago
    I try to, but nosql discussions get pretty heated.
    - dang 11y ago
      Lots of discussions do. The trick is not to post while heated, or at least to edit the excesses out.
- true_religion 11y ago
  Does CouchDB actually store the data in JSON format on disk? Or does it use a more specialized format? Elasticsearch, for example, has a full featured JSON API, but stores documents on disk in Lucerne---not that you'd ever know that if you only perused the API.
  - skjhn 11y ago
    I'm not sure about the format, but it's compressed.
- skjhn 11y ago
  Let's see if I can help here. A lot people like async map-reduce. If you need to perform aggregation on a lot of data, its constantly growing, and you need the results to be current, async map-reduce is great. In the best case scenario, the results are precomputed. In a worst case scenario, they are a few seconds out of date. However, you have the option of forcing an update if need be. Either way, it's a hell of lot faster than running the full aggregation every time it's requested. Redis is great, but a) the memcached protocol is well established and b) Redis is more than a simple cache. BSON vs. JSON, what's the point here? A query doesn't have to hit every index node. That doesn't make any sense. In fact, it's quite the opposite. With local indexes, you would, in fact, hit every single node. With global secondary indexes, you hit the index node with the right index. Are you talking about partial updates? If so, yes, that will be available in the next developer preview. Stay tuned.
  - ddorian43 11y ago
    Hi, 1. For indexes ? But only couchdb has them. If more people would liked them it would be more popular ? 2. Yeah. I agree that for distributed-persistent-memcache it's good. 2.5 Json is inefficient. 3. Yeah, but you usually shard indexes, say by user_id. So when you're filtering where user_id=x and column_b=y you hit only 1 node. 4. Things that don't have partial-updates are key-value dbs, right ? If yes, why don't you call yourself that till you actually have partial-updates ?
    - skjhn 11y ago
      There are a handful of databases that implement map-reduce one way or another - CouchDB, Couchbase, and MongoDB off the top of my head. Views might be a CouchDB/Couchbase concept, but not incremental map-reduce. In what way is JSON inefficient? Are we talking about size? GSI indexes may or may not be partitioned. With GSI, depending on the index size and resources available, you would most likely NOT partition the index - that's the recommendation. You can create an index on user_id and column_b, place it on a specific node, and you'd only be hitting that node for a query. Especially if it's a covering index. Again, databases without GSI indexes have an index partition on every single node - that means hitting every single one for every single query. I'm still not sure what you're trying to get at. I'm guessing you are referring to MongoDB shards and routers. However, that example doesn't make sense. If user_id is the shard key, then yes, the router sends the query to the right node. The same thing happens with Couchbase. Given the key, you get the document straight from the node that has it. However, if you have user_id, why are you querying on column_b too? Now, if user_id is not the shard key, then no, the router does not send the query to the right node, its sends the query to every single node. I'm generalizing, but key-value databases are best for key-value operations on arbitrary data. Document databases understand JSON and, as such, can provide access via queries. With Couchbase, you can choose from views, N1QL (SQL), geospatial (built on views), or full-text search (preview). Pretty far off from a key-value store. That, and it already has support for partial updates via N1QL. However, my assumption was that you were talking about partial updates via key-value operations.
strmpnk 11y ago
Couchbase may have involved the creators of Apache CouchDB but they are only lightly related these days (mostly some remnants of replication and map-reduce concepts). Both have move some distance since then so it's best to assume that CouchDB != Couchbase. (A most annoying name clash but that's that.) What is being talked about here is the 2.0 release which is under development. It integrates IBM Cloudant's clustering layer which was previously called BigCouch. It's been a long road but it's good to see the C in couch (Cluster Of Unreliable Commodity Hardware) finally get supported. IMO, the HTTP API code in the project is probably the biggest jungle left mostly untouched from the earlier days of CouchDB and is ripe for a major cleanup. It's not surprising to see bottlenecks like this and it's great to see the author find a reasonable fix for the time being.
Zikes 11y ago
There is no "objectively best", only "that which is best suited to the problem at hand".
- hnbroseph 11y ago
  > that which is best suited to the problem at hand also best suited...according to our understanding of the problem space, our capability/competency to tackle it, our resources presently available to allocate, our worldview and preferences, any external constraints such as customer requirements, etc, etc, etc.
true_religion 11y ago
> Erlang is one of those languages that is objectively superior -- I've yet to meet another language that does concurrency right-- yet many hackers just ignore it because it's not got java's syntax. Which is silly. What do you think about Elixir?
lackbeard 11y ago
I wonder why every CouchDB-related post to Hacker News has someone in the comments derailing things by bringing up Couchbase...
workusername 11y ago
This is the best NoSQL database going. Considering how different the so-called NoSQL data stores are, and how uselessly broad the NoSQL categorisation is, I'd be interested to hear the comparison points.

by MCRed 11y ago

ddorian43 11y ago

*This is the best NoSQL database going. Yeah, right. Most of the things about it ARE wrong: 1. couchdb views, who likes async map-reduce indexes ? 2. memcached built ok (better would be redis built in) 3. json documents, even mongodb has bson and not json 4. the new global-indexes-thing IS NOT scalable because you have to hit every index-node to do a query 5. when will you be able to modify a document ? looks like still in beta

dang 11y ago
> Yeah, right. Most of the things about it ARE wrong You've clearly got some good points here, but please don't break the HN guidelines by introducing them that way.
- ddorian43 11y ago
  I try to, but nosql discussions get pretty heated.
  - dang 11y ago
    Lots of discussions do. The trick is not to post while heated, or at least to edit the excesses out.
true_religion 11y ago
Does CouchDB actually store the data in JSON format on disk? Or does it use a more specialized format? Elasticsearch, for example, has a full featured JSON API, but stores documents on disk in Lucerne---not that you'd ever know that if you only perused the API.
- skjhn 11y ago
  I'm not sure about the format, but it's compressed.
skjhn 11y ago
Let's see if I can help here. A lot people like async map-reduce. If you need to perform aggregation on a lot of data, its constantly growing, and you need the results to be current, async map-reduce is great. In the best case scenario, the results are precomputed. In a worst case scenario, they are a few seconds out of date. However, you have the option of forcing an update if need be. Either way, it's a hell of lot faster than running the full aggregation every time it's requested. Redis is great, but a) the memcached protocol is well established and b) Redis is more than a simple cache. BSON vs. JSON, what's the point here? A query doesn't have to hit every index node. That doesn't make any sense. In fact, it's quite the opposite. With local indexes, you would, in fact, hit every single node. With global secondary indexes, you hit the index node with the right index. Are you talking about partial updates? If so, yes, that will be available in the next developer preview. Stay tuned.
- ddorian43 11y ago
  Hi, 1. For indexes ? But only couchdb has them. If more people would liked them it would be more popular ? 2. Yeah. I agree that for distributed-persistent-memcache it's good. 2.5 Json is inefficient. 3. Yeah, but you usually shard indexes, say by user_id. So when you're filtering where user_id=x and column_b=y you hit only 1 node. 4. Things that don't have partial-updates are key-value dbs, right ? If yes, why don't you call yourself that till you actually have partial-updates ?
  - skjhn 11y ago
    There are a handful of databases that implement map-reduce one way or another - CouchDB, Couchbase, and MongoDB off the top of my head. Views might be a CouchDB/Couchbase concept, but not incremental map-reduce. In what way is JSON inefficient? Are we talking about size? GSI indexes may or may not be partitioned. With GSI, depending on the index size and resources available, you would most likely NOT partition the index - that's the recommendation. You can create an index on user_id and column_b, place it on a specific node, and you'd only be hitting that node for a query. Especially if it's a covering index. Again, databases without GSI indexes have an index partition on every single node - that means hitting every single one for every single query. I'm still not sure what you're trying to get at. I'm guessing you are referring to MongoDB shards and routers. However, that example doesn't make sense. If user_id is the shard key, then yes, the router sends the query to the right node. The same thing happens with Couchbase. Given the key, you get the document straight from the node that has it. However, if you have user_id, why are you querying on column_b too? Now, if user_id is not the shard key, then no, the router does not send the query to the right node, its sends the query to every single node. I'm generalizing, but key-value databases are best for key-value operations on arbitrary data. Document databases understand JSON and, as such, can provide access via queries. With Couchbase, you can choose from views, N1QL (SQL), geospatial (built on views), or full-text search (preview). Pretty far off from a key-value store. That, and it already has support for partial updates via N1QL. However, my assumption was that you were talking about partial updates via key-value operations.

strmpnk 11y ago

Couchbase may have involved the creators of Apache CouchDB but they are only lightly related these days (mostly some remnants of replication and map-reduce concepts). Both have move some distance since then so it's best to assume that CouchDB != Couchbase. (A most annoying name clash but that's that.) What is being talked about here is the 2.0 release which is under development. It integrates IBM Cloudant's clustering layer which was previously called BigCouch. It's been a long road but it's good to see the C in couch (Cluster Of Unreliable Commodity Hardware) finally get supported. IMO, the HTTP API code in the project is probably the biggest jungle left mostly untouched from the earlier days of CouchDB and is ripe for a major cleanup. It's not surprising to see bottlenecks like this and it's great to see the author find a reasonable fix for the time being.

Zikes 11y ago

There is no "objectively best", only "that which is best suited to the problem at hand".

hnbroseph 11y ago
> that which is best suited to the problem at hand also best suited...according to our understanding of the problem space, our capability/competency to tackle it, our resources presently available to allocate, our worldview and preferences, any external constraints such as customer requirements, etc, etc, etc.

true_religion 11y ago

> Erlang is one of those languages that is objectively superior -- I've yet to meet another language that does concurrency right-- yet many hackers just ignore it because it's not got java's syntax. Which is silly. What do you think about Elixir?

lackbeard 11y ago

I wonder why every CouchDB-related post to Hacker News has someone in the comments derailing things by bringing up Couchbase...

workusername 11y ago

This is the best NoSQL database going. Considering how different the so-called NoSQL data stores are, and how uselessly broad the NoSQL categorisation is, I'd be interested to hear the comparison points.