9 ms·

Honest question, is MongoDB actually scalable? I keep reading about how it is not for serious business[1][2][3][4]. Who is using Mongo at scale and what sort of

Honest question, is MongoDB actually scalable? I keep reading about how it is not for serious business[1][2][3][4]. Who is using Mongo at scale and what sort of loads are they seeing and are the level of problems on par with other solutions? As a developer I like Mongo's API, but I would worry about using it given what has been written about it. [1] http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/284-call-me-maybe-mongodb [2] http://svs.io/post/31724990463/why-i-migrated-away-from-mongodb http://svs.io/post/31724990463/why-i-migrated-away-from-mong... [3] http://use-the-index-luke.com/blog/2013-10/mysql-is-to-sql-like-mongodb-to-nosql http://use-the-index-luke.com/blog/2013-10/mysql-is-to-sql-l... [4] https://news.ycombinator.com/item?id=8353228 https://news.ycombinator.com/item?id=8353228

x0x0 12y ago
No, and even modest write loads will cause immense pain -- there was a single lock per db when I last used them. Plus lots of the standard tricks for getting performance out of dbs don't work for them. I'd recommend staying away unless your performance needs are very moderate and you must have unstructured tables.
- mcot2 12y ago
  Check out tokumx, it fixes a lot of the problems with mongo.
  - cheald 12y ago
    Seconded; we've had a very positive experience with it so far. We're running around 1.5TB in it (and growing) and it's working pretty decently. Our biggest problems with it have been CPU saturation due to compression (solvable with sharding) and oplog size (due to supporting ACID; supposedly much better in the upcoming release), but both of those are surmountable. In exchange we get massively better disk usage characteristics, no global locks, ACID compliance, transactions, and generally better performance. It's not perfect, but it solved a lot of our problems.
- rogerbinns 12y ago
  There is still a single lock per db. It also uses huge amounts of space (around double JSON of same data). That puts increased pressure on memory. It also has no usable compaction. And can be rather dumb with indexes. (note) The main mongo solution for things is to run lots of copies of it across many machines each with small chunks of data. In theory what is then a pain point on bigger systems becomes lots of lesser pains on small systems. (note) every criticism is answered with how the next release will improve things. Sometimes that happens.
deepsun 12y ago
I agree with you, per our experience Mongo is not scalable. We use Mongo in our company (60GB per db.stats().fileSize), both for relational data, and for what we call "fat data" (just to avoid calling these mere gigabytes "big data" -- those are just lots of statistical numbers, more like OLAP cubes). Over many problems and sad surprises, we came to the point that we'd better have used PostgreSQL (or any other SQL) for relational data, and some NoSQL for "fat data". Or just single PostgreSQL would do better, because Mongo's feature set is a subset of PostgreSQL's feature set. Just Mongo renamed them and pretends it's something new.
- donw 12y ago
  This fits with my experience as well, with (possibly) a larger data set. MongoDB is actually pretty good as a document store, where you can accept soft commits, and are dealing with non-relational data that doesn't need to be aggregated. When you start needing more than just a loose pile of documents, or live in a world where you really need ACID, Mongo has this nasty habit of falling down on the floor and twitching. Those problems are solveable, but it doesn't just happen "out-of-the-box".
  - e12e 12y ago
    I wonder, what do you get over exporting a file system over webdav, if you only do "document storage" of json documents? After all filesystems are made for storing a hierarchy of files... some indexes for searching? More convenient rest api?
    - donw 12y ago
      The API is much more convenient, and last time I checked, WebDAV didn't provide the same wealth of query options or multi-node replication. Plus, I can get solid commercial support for MongoDB, which matters a lot if you're using it to power a business.
      - e12e 12y ago
        Multi-node replication for webdav would be done at the file system level (eg: gfs or whatnot). By query options, to you mean some kind of join, for example? Obviously anything beyond plain id-based get/set (by file path/url) isn't possible with "just" webdav. But if you already limit yourself to looking at single documents (ie: json-files) -- you could always fetch doc1, find the id/path of doc2 in doc1 and fetch that. I'm not saying this sounds like a sane db strategy, I'm just not convinced doing the same thing, with a little easier api sounds like a sane strategy either?
notastartup 12y ago
Agreed but maybe it was because at a startup I worked for a few years ago used Meteor.
ub 12y ago
My experience with MongoDB has been terrible. Apart from just look-ups I don't think it's meant for much data wrangling. Joins with different collections are harder to do. I see the best use case of Mongo is for data dumps.
- tracker1 12y ago
  It's pretty good for a document store.. partial updates to documents, as well as indexing work well. Setting it up for replica sets with auto failover is much easier than say PostgreSQL, as is the API interface (especially geo searches). It's run well for most of my own uses with it, though I do keep an eye towards RethinkDB as well as ElasticSearch and Cassandra. RethinkDB really needs to get the auto-hot failover and geo searches worked out, geo is on the table for the next release iirc, and failover the next after. Cassandra is great for key/value searches, but falls down for range queries. ElasticSearch is pretty awesome in its' own right, but not perfect either. PostgreSQL has a lot to offer as well. 9.4 should be pretty sweet, and if they get automagic failover in the community versions worked out, I'm totally in. It really just depends on what your workload is... MongoDB offers a lot of DB-like scenarios against indexes in a single collection, a clean set of interfaces, and a fairly responsive system overall. There have been growing pains, and problems... the same can be said of any database. To each their own, it really just depends on your needs, and for that matter how far out your project's release is, vs. how long you need to support it. Right now, I'm replacing an over-normalized SQL database structure that is pretty rancid. Most of the data fits so much better with a document db it isn't funny. When I had done the first parts, I had issues with geo searches in similar alternatives, and that has been a deal breaker for a lot of the options. You don't use a document store if you need to use joins.. you're better off either duplicating the data, or using separate on-demand queries... odds are your data isn't shaped right and you should have used a structured database, or you aren't thinking of the problem right. YMMV.
  - illumen 12y ago
    You may not need joins, but someone else May. Someone else likely Will in my experience. Or I will, when a new requirement comes in.
aikah 12y ago
Mongodb is great ... for logging stuff,or quick prototyping.It's not that fast on writes ,but pretty fast on reads. In my opinion,what people usually really want is a RDBMS + a full text search engine like elastic search. But again,one needs to set these things up. Mongodb didnt have aggregating features in the past,and their map/reduce feature is not that good.But again,the product is still young,maybe it will get better.

x0x0 12y ago

No, and even modest write loads will cause immense pain -- there was a single lock per db when I last used them. Plus lots of the standard tricks for getting performance out of dbs don't work for them. I'd recommend staying away unless your performance needs are very moderate and you must have unstructured tables.

mcot2 12y ago
Check out tokumx, it fixes a lot of the problems with mongo.
- cheald 12y ago
  Seconded; we've had a very positive experience with it so far. We're running around 1.5TB in it (and growing) and it's working pretty decently. Our biggest problems with it have been CPU saturation due to compression (solvable with sharding) and oplog size (due to supporting ACID; supposedly much better in the upcoming release), but both of those are surmountable. In exchange we get massively better disk usage characteristics, no global locks, ACID compliance, transactions, and generally better performance. It's not perfect, but it solved a lot of our problems.
rogerbinns 12y ago
There is still a single lock per db. It also uses huge amounts of space (around double JSON of same data). That puts increased pressure on memory. It also has no usable compaction. And can be rather dumb with indexes. (note) The main mongo solution for things is to run lots of copies of it across many machines each with small chunks of data. In theory what is then a pain point on bigger systems becomes lots of lesser pains on small systems. (note) every criticism is answered with how the next release will improve things. Sometimes that happens.

deepsun 12y ago

I agree with you, per our experience Mongo is not scalable. We use Mongo in our company (60GB per db.stats().fileSize), both for relational data, and for what we call "fat data" (just to avoid calling these mere gigabytes "big data" -- those are just lots of statistical numbers, more like OLAP cubes). Over many problems and sad surprises, we came to the point that we'd better have used PostgreSQL (or any other SQL) for relational data, and some NoSQL for "fat data". Or just single PostgreSQL would do better, because Mongo's feature set is a subset of PostgreSQL's feature set. Just Mongo renamed them and pretends it's something new.

donw 12y ago
This fits with my experience as well, with (possibly) a larger data set. MongoDB is actually pretty good as a document store, where you can accept soft commits, and are dealing with non-relational data that doesn't need to be aggregated. When you start needing more than just a loose pile of documents, or live in a world where you really need ACID, Mongo has this nasty habit of falling down on the floor and twitching. Those problems are solveable, but it doesn't just happen "out-of-the-box".
- e12e 12y ago
  I wonder, what do you get over exporting a file system over webdav, if you only do "document storage" of json documents? After all filesystems are made for storing a hierarchy of files... some indexes for searching? More convenient rest api?
  - donw 12y ago
    The API is much more convenient, and last time I checked, WebDAV didn't provide the same wealth of query options or multi-node replication. Plus, I can get solid commercial support for MongoDB, which matters a lot if you're using it to power a business.
    - e12e 12y ago
      Multi-node replication for webdav would be done at the file system level (eg: gfs or whatnot). By query options, to you mean some kind of join, for example? Obviously anything beyond plain id-based get/set (by file path/url) isn't possible with "just" webdav. But if you already limit yourself to looking at single documents (ie: json-files) -- you could always fetch doc1, find the id/path of doc2 in doc1 and fetch that. I'm not saying this sounds like a sane db strategy, I'm just not convinced doing the same thing, with a little easier api sounds like a sane strategy either?

notastartup 12y ago

Agreed but maybe it was because at a startup I worked for a few years ago used Meteor.

ub 12y ago

My experience with MongoDB has been terrible. Apart from just look-ups I don't think it's meant for much data wrangling. Joins with different collections are harder to do. I see the best use case of Mongo is for data dumps.

tracker1 12y ago
It's pretty good for a document store.. partial updates to documents, as well as indexing work well. Setting it up for replica sets with auto failover is much easier than say PostgreSQL, as is the API interface (especially geo searches). It's run well for most of my own uses with it, though I do keep an eye towards RethinkDB as well as ElasticSearch and Cassandra. RethinkDB really needs to get the auto-hot failover and geo searches worked out, geo is on the table for the next release iirc, and failover the next after. Cassandra is great for key/value searches, but falls down for range queries. ElasticSearch is pretty awesome in its' own right, but not perfect either. PostgreSQL has a lot to offer as well. 9.4 should be pretty sweet, and if they get automagic failover in the community versions worked out, I'm totally in. It really just depends on what your workload is... MongoDB offers a lot of DB-like scenarios against indexes in a single collection, a clean set of interfaces, and a fairly responsive system overall. There have been growing pains, and problems... the same can be said of any database. To each their own, it really just depends on your needs, and for that matter how far out your project's release is, vs. how long you need to support it. Right now, I'm replacing an over-normalized SQL database structure that is pretty rancid. Most of the data fits so much better with a document db it isn't funny. When I had done the first parts, I had issues with geo searches in similar alternatives, and that has been a deal breaker for a lot of the options. You don't use a document store if you need to use joins.. you're better off either duplicating the data, or using separate on-demand queries... odds are your data isn't shaped right and you should have used a structured database, or you aren't thinking of the problem right. YMMV.
- illumen 12y ago
  You may not need joins, but someone else May. Someone else likely Will in my experience. Or I will, when a new requirement comes in.

aikah 12y ago

Mongodb is great ... for logging stuff,or quick prototyping.It's not that fast on writes ,but pretty fast on reads. In my opinion,what people usually really want is a RDBMS + a full text search engine like elastic search. But again,one needs to set these things up. Mongodb didnt have aggregating features in the past,and their map/reduce feature is not that good.But again,the product is still young,maybe it will get better.