5 ms·

MongoDB Rules Single Node Deployments, Fails to Scale

42 points by skjhn 12y ago

jchrisa 12y ago
"This is a sweet spot for MongoDB." Made my day. Full disclosure -- We've been locking horns with them for a while now, so of course this feels good.
ddorian43 12y ago
Couchbase, the nodejs of databases. Everything is async. Want indexes updated, async. Want replication, async. Want commiting stuff to disk, async. When you try to sync stuff (indexes + disk-commit) the performance will take a hit.
- jchrisa 12y ago
  Many applications find sync writes to replica memory on other rack(s) to be a good substitute for sync to disk. The disk sync can be batched every second or so for higher throughput and your application is safe. Where this really plays out well is when the disk goes slow for a minute (cloud-style) and the database stays fast.
skjhn 12y ago
TLDR: MongoDB benchmarked Couchbase Server by having it perform CAS operations (read + update) while MongoDB performed basic operations (update), they used an outdated client library that is two years old, and they performed it with single node deployments.
- codebeaker 12y ago
  MongoDB showing that in the years since this[1] was published that have learned nothing. Still chasing artificial benchmarks at the expensive of data safety and either personal or corporate integrity. [1]: http://pastebin.com/raw.php?i=FD3xe6Jt http://pastebin.com/raw.php?i=FD3xe6Jt
  - towelguy 12y ago
    Does this still applies to MongoDB 3.0? since it seems to be a rewrite with a new storage engine [1]. [1] http://www.mongodb.com/blog/post/announcing-mongodb-30 http://www.mongodb.com/blog/post/announcing-mongodb-30
    - nemo44x 12y ago
      That is based on a pre-2.0 version of MongoDB. The changes and vast improvements from that time to now are huge. MongoDB probably got a bit ahead of itself too fast - it wasn't mature and took a lot of development shortcuts to get features out fast and people started using it like a mature datastore too fast - and the company behind it didn't help either. But, without a doubt there have been vast improvements across the board and I give them credit for spending the last year and a half not adding many new features but rebuilding the codebase, making it extendable (pluggable storage engines is a cool thing) and addressing numerous issues. It still has some issues (b-trees aren't the best implementation imo) but the codebase is in a state where I expect over the next few years it will be quite reliable. The tooling around MongoDB has gotten better too. There's still a ton of ways to shoot yourself in the foot too but the defaults are more sensible I think now than then. What they got right is the clients are awesome and the query language is powerful - especially updates. This matters for a lot of people - it isn't just a drag race always in terms of performance. Now, with increased reliability it's becoming something usable for some use cases.
  - functional_test 12y ago
    I reply to these posts on HN relatively frequently these days. These sorts of accusations about Mongo are out of hand, and not congruent with reality. I say this as someone who used MongoDB extensively. I don't have time to address all of the problems, but let's look at #1 on his list: 1. MongoDB issues writes in unsafe ways by default in order to win benchmarks This is objectively false. At the time, Mongo chose to turn off write confirmations by default -- this was clearly documented in many places including introductory documentation. Surely no one would have deployed a database without even a cursory skim of the docs, right? The notion that it was done to win benchmarks is baseless. Furthermore, in response to the community's reaction, the default was changed. This is a recurring problem with the complaints about Mongo. You cannot use ad copy to evaluate a database (or any product for that matter, but definitely not something as important as your primary data store). You must read the documentation. You must do a trial run and make sure it will meet the needs of your use case. You can't be upset about the results you didn't get from the work you didn't do.
- mushi 12y ago
  isn't it because mongo has update operation that can do a $set (like SQL) and couchbase doesn't? how else can couchbase update a field in an existing document?
  - deleted 12y ago
    [deleted]
  - KMag 12y ago
    The "CAS" in the OP stands for "Compare and Swap". The complaint is that they had Couchbase checking for read-write conflicts while Mongo was doing blind updates regardless of if the data in Mongo was different from what the client expected. The complaint isn't about sub-document updates vs. full document updates. The complaint is about conflict checking. Couchbase could have been told to blindly write, but wasn't, and Mongo was told to blindly write.
stingraycharles 12y ago
The submission title is incomplete; the full title continues with ".. but fails to scale", which is important.
- skjhn 12y ago
  Good catch. Fixed.
nullspace 12y ago
I did not see a link to the independent benchmarks that was referenced here. Does anyone have the link? I would be interested in seeing the actual results and summary of it.
- jchrisa 12y ago
  They promoted it the other day: http://www.informationweek.com/big-data/software-platforms/mongodb-counters-couchbase-performance-claims/a/d-id/1319696 http://www.informationweek.com/big-data/software-platforms/m...
616c 12y ago
I guess it is ... nice to see Couchbase again. I will not speak about them because I know little. They did open source their components, but in dumping ground style without full instrumentation, meaning they had inline instructions with a bunch of shell commands. I checked before submitting this, and it appears they have cleaned it up a lot, at least using the Android repo tool apparently. I will have to check it out. http://www.couchbase.com/open-source http://www.couchbase.com/open-source But does anyone use Apache CouchDB? I still do see it as one of the more unique NoSQL systems. I specifically loved the idea of CouchApps, and using it a neat version-controlled JS web app hosting system and document repository. I have obviously lost track over the years. Does anyone use CouchDB seriously? Can they mention if the same raggging about giant irrecoverable disk consumption and other gripes still apply? Did anyone consider CouchDB and stick with Couchbase and can explain why?
- T-A 12y ago
  Maybe I'm misreading your question, but anyway, to prevent misunderstandings: Couchbase and CouchDB are different animals. http://www.couchbase.com/couchbase-vs-couchdb http://www.couchbase.com/couchbase-vs-couchdb
  - 616c 12y ago
    Maybe I am mistaken, but was Couchbase not forked by one of the original CouchDB developers, with hot paths of code rewritten in C and employing memcache to increase the speed? http://www.couchbase.com/couchbase-vs-couchdb http://www.couchbase.com/couchbase-vs-couchdb http://damienkatz.net/2012/01/the_future_of_couchdb.html http://damienkatz.net/2012/01/the_future_of_couchdb.html (An announcement from one of the CouchDB -> Couchbase developers) EDIT: I think I confused blog posts. That is a guy involved in Couchbase, no? I forget who but one of the other CouchDB only devs who worked with him was pissed and wrote a rantish piece. Too tired to find it now.
- jchrisa 12y ago
  CouchDB is growing and maturing. As far as Couchbase development goes, we use the Apache CouchDB protocol in our mobile products (and interoperate w the whole ecosystem including PouchDB.) Our iOS Android Windows and Golang database implementations are all open source. http://developer.couchbase.com/mobile/ http://developer.couchbase.com/mobile/
randiantech 12y ago
MongoDB is too much mainstream. Thats unnaceptable i guess.
arthursilva 12y ago
I'm predicting Couchbase 4.0 eating a fair share of Mongodb cake.
stonewhite 12y ago
Isn't this like, beating the dead horse a bit?
wheaties 12y ago
Ok, so they used an outdated client and then used a double operation on Couchbase when comparing to MongoDB. The other part of "doesn't scale" wasn't addressed at all. I hate MongoDB with a passion (burned too many times) but even this is just FUD-like without numbers or something to back it up. C'mon, you guys can do better than that. Just working is enough to show up Mongo. Seriously, the bar isn't that high.
- jchrisa 12y ago
  Here's the benchmark MongoDB was replying to with their benchmark: http://news.avalonconsult.com/2015/03/19/performance_benchmark/ http://news.avalonconsult.com/2015/03/19/performance_benchma... TLDR: for highly concurrent workloads on a nine node cluster, Couchbase Server handles way more traffic than MongoDB without response times slowing down.
mushi 12y ago
I suppose couchbase would prefer benchmarks like their previous ones where they overwrote the full ten field document with a single new field since they don't have an update operation.
scalesolved 12y ago
I like and have used both in production, they both have use cases where they absolutely excel and blow away the competition. It's kind of sad to see both sides slinging mud, we're all adults and we are able to choose a technology from our own benchmarks and experiments. It kind of puts me off of both of them.
- jchrisa 12y ago
  The sad reality is that a little dust-up is probably good for business for both of us. For me (and probably a bunch of folks on the Mongo side) it's all in good fun.

42 points by skjhn 12y ago

"This is a sweet spot for MongoDB." Made my day. Full disclosure -- We've been locking horns with them for a while now, so of course this feels good.

ddorian43 12y ago

Couchbase, the nodejs of databases. Everything is async. Want indexes updated, async. Want replication, async. Want commiting stuff to disk, async. When you try to sync stuff (indexes + disk-commit) the performance will take a hit.

jchrisa 12y ago
Many applications find sync writes to replica memory on other rack(s) to be a good substitute for sync to disk. The disk sync can be batched every second or so for higher throughput and your application is safe. Where this really plays out well is when the disk goes slow for a minute (cloud-style) and the database stays fast.

skjhn 12y ago

TLDR: MongoDB benchmarked Couchbase Server by having it perform CAS operations (read + update) while MongoDB performed basic operations (update), they used an outdated client library that is two years old, and they performed it with single node deployments.

codebeaker 12y ago
MongoDB showing that in the years since this[1] was published that have learned nothing. Still chasing artificial benchmarks at the expensive of data safety and either personal or corporate integrity. [1]: http://pastebin.com/raw.php?i=FD3xe6Jt http://pastebin.com/raw.php?i=FD3xe6Jt
- towelguy 12y ago
  Does this still applies to MongoDB 3.0? since it seems to be a rewrite with a new storage engine [1]. [1] http://www.mongodb.com/blog/post/announcing-mongodb-30 http://www.mongodb.com/blog/post/announcing-mongodb-30
  - nemo44x 12y ago
    That is based on a pre-2.0 version of MongoDB. The changes and vast improvements from that time to now are huge. MongoDB probably got a bit ahead of itself too fast - it wasn't mature and took a lot of development shortcuts to get features out fast and people started using it like a mature datastore too fast - and the company behind it didn't help either. But, without a doubt there have been vast improvements across the board and I give them credit for spending the last year and a half not adding many new features but rebuilding the codebase, making it extendable (pluggable storage engines is a cool thing) and addressing numerous issues. It still has some issues (b-trees aren't the best implementation imo) but the codebase is in a state where I expect over the next few years it will be quite reliable. The tooling around MongoDB has gotten better too. There's still a ton of ways to shoot yourself in the foot too but the defaults are more sensible I think now than then. What they got right is the clients are awesome and the query language is powerful - especially updates. This matters for a lot of people - it isn't just a drag race always in terms of performance. Now, with increased reliability it's becoming something usable for some use cases.
- functional_test 12y ago
  I reply to these posts on HN relatively frequently these days. These sorts of accusations about Mongo are out of hand, and not congruent with reality. I say this as someone who used MongoDB extensively. I don't have time to address all of the problems, but let's look at #1 on his list: 1. MongoDB issues writes in unsafe ways by default in order to win benchmarks This is objectively false. At the time, Mongo chose to turn off write confirmations by default -- this was clearly documented in many places including introductory documentation. Surely no one would have deployed a database without even a cursory skim of the docs, right? The notion that it was done to win benchmarks is baseless. Furthermore, in response to the community's reaction, the default was changed. This is a recurring problem with the complaints about Mongo. You cannot use ad copy to evaluate a database (or any product for that matter, but definitely not something as important as your primary data store). You must read the documentation. You must do a trial run and make sure it will meet the needs of your use case. You can't be upset about the results you didn't get from the work you didn't do.
mushi 12y ago
isn't it because mongo has update operation that can do a $set (like SQL) and couchbase doesn't? how else can couchbase update a field in an existing document?
- deleted 12y ago
  [deleted]
- KMag 12y ago
  The "CAS" in the OP stands for "Compare and Swap". The complaint is that they had Couchbase checking for read-write conflicts while Mongo was doing blind updates regardless of if the data in Mongo was different from what the client expected. The complaint isn't about sub-document updates vs. full document updates. The complaint is about conflict checking. Couchbase could have been told to blindly write, but wasn't, and Mongo was told to blindly write.

stingraycharles 12y ago

The submission title is incomplete; the full title continues with ".. but fails to scale", which is important.

skjhn 12y ago
Good catch. Fixed.

nullspace 12y ago

I did not see a link to the independent benchmarks that was referenced here. Does anyone have the link? I would be interested in seeing the actual results and summary of it.

jchrisa 12y ago
They promoted it the other day: http://www.informationweek.com/big-data/software-platforms/mongodb-counters-couchbase-performance-claims/a/d-id/1319696 http://www.informationweek.com/big-data/software-platforms/m...

616c 12y ago

I guess it is ... nice to see Couchbase again. I will not speak about them because I know little. They did open source their components, but in dumping ground style without full instrumentation, meaning they had inline instructions with a bunch of shell commands. I checked before submitting this, and it appears they have cleaned it up a lot, at least using the Android repo tool apparently. I will have to check it out. http://www.couchbase.com/open-source http://www.couchbase.com/open-source But does anyone use Apache CouchDB? I still do see it as one of the more unique NoSQL systems. I specifically loved the idea of CouchApps, and using it a neat version-controlled JS web app hosting system and document repository. I have obviously lost track over the years. Does anyone use CouchDB seriously? Can they mention if the same raggging about giant irrecoverable disk consumption and other gripes still apply? Did anyone consider CouchDB and stick with Couchbase and can explain why?

T-A 12y ago
Maybe I'm misreading your question, but anyway, to prevent misunderstandings: Couchbase and CouchDB are different animals. http://www.couchbase.com/couchbase-vs-couchdb http://www.couchbase.com/couchbase-vs-couchdb
- 616c 12y ago
  Maybe I am mistaken, but was Couchbase not forked by one of the original CouchDB developers, with hot paths of code rewritten in C and employing memcache to increase the speed? http://www.couchbase.com/couchbase-vs-couchdb http://www.couchbase.com/couchbase-vs-couchdb http://damienkatz.net/2012/01/the_future_of_couchdb.html http://damienkatz.net/2012/01/the_future_of_couchdb.html (An announcement from one of the CouchDB -> Couchbase developers) EDIT: I think I confused blog posts. That is a guy involved in Couchbase, no? I forget who but one of the other CouchDB only devs who worked with him was pissed and wrote a rantish piece. Too tired to find it now.
jchrisa 12y ago
CouchDB is growing and maturing. As far as Couchbase development goes, we use the Apache CouchDB protocol in our mobile products (and interoperate w the whole ecosystem including PouchDB.) Our iOS Android Windows and Golang database implementations are all open source. http://developer.couchbase.com/mobile/ http://developer.couchbase.com/mobile/

randiantech 12y ago

MongoDB is too much mainstream. Thats unnaceptable i guess.

arthursilva 12y ago

I'm predicting Couchbase 4.0 eating a fair share of Mongodb cake.

stonewhite 12y ago

Isn't this like, beating the dead horse a bit?

wheaties 12y ago

Ok, so they used an outdated client and then used a double operation on Couchbase when comparing to MongoDB. The other part of "doesn't scale" wasn't addressed at all. I hate MongoDB with a passion (burned too many times) but even this is just FUD-like without numbers or something to back it up. C'mon, you guys can do better than that. Just working is enough to show up Mongo. Seriously, the bar isn't that high.

jchrisa 12y ago
Here's the benchmark MongoDB was replying to with their benchmark: http://news.avalonconsult.com/2015/03/19/performance_benchmark/ http://news.avalonconsult.com/2015/03/19/performance_benchma... TLDR: for highly concurrent workloads on a nine node cluster, Couchbase Server handles way more traffic than MongoDB without response times slowing down.

mushi 12y ago

I suppose couchbase would prefer benchmarks like their previous ones where they overwrote the full ten field document with a single new field since they don't have an update operation.

scalesolved 12y ago

I like and have used both in production, they both have use cases where they absolutely excel and blow away the competition. It's kind of sad to see both sides slinging mud, we're all adults and we are able to choose a technology from our own benchmarks and experiments. It kind of puts me off of both of them.

jchrisa 12y ago
The sad reality is that a little dust-up is probably good for business for both of us. For me (and probably a bunch of folks on the Mongo side) it's all in good fun.