4 ms·

3 reasons to use MongoDB

35 points by angilly 16y ago

mrkurt 16y ago
Really, the #1 reason to use MongoDB (if you're me, anyway) is to save development time associated with making your relational schema start small and change as your new app progresses. I feel a smug sense of joy every time I add a field somewhere, or delete another, or create some kind of nested document. It's taken me a while to really understand how many compromises I used to make because changing schemas is a pain in the ass. Simplified queries, though, are a knock against mongo. Joins are great and I would like to do joins on my Mongo documents, but I end up having to replicate a lot of that in code. Sure it's nice that a document can be more complex and you don't spend a lot of time moving things into tables that are really part of the same record. It's nice because it's not forced, though, not because keeping data in different tables is always the wrong way to do things.
- weixiyen 16y ago
  A lot sites these days don't do joins with relational databases anyways. They just use mysql or postgresql as a key value store.
- saurik 16y ago
  I use PostgreSQL and change my schema constantly, adding/removing columns, changing data types, switching around foreign key constraints, all within safely guarded transactions that I can roll back if I realize that I'm doing something wrong. (And yes: adding/removing/renaming columns is "instant": it doesn't actually do the work of rewriting existing rows on disk.) Frankly, I also use MongoDB, and I'm terrified of screwing much with the schema, because then I either a) have to make certain I have anal documentation about what fields are in use on what subsets of objects, keeping code around to make certain to detect and interpret old kinds of data, or b) use "simplified queries" (really, write a bunch of manual code as if I didn't have a query model at all) in order to find and update these old objects, non-atomically and with no transaction safety. Seriously: the only reason I've so far heard for why having dynamic type verification in my database server is valuable is when you have /so much data/ that it is now fundamentally infeasible to make changes to it with any centralized transaction control--specifically, Google's "it's always the interim somewhere" scenario--not because it is somehow more convenient to do so when you only have in the tens of millions of rows.
ergo98 16y ago
MongoDB is elegant and remarkably powerful. Every developer should run through the excellent getting started tutorial they have for it, as it really is eye opening. http://www.mongodb.org/display/DOCS/Tutorial http://www.mongodb.org/display/DOCS/Tutorial However I have to respectfully disagree on the "Simple queries" bit. The SQL example given is kind of terrible, however how about- SELECT * FROM users WHERE id IN (SELECT user_id FROM events WHERE published_at IS NOT NULL) or SELECT * FROM users WHERE EXISTS (SELECT 1 FROM events WHERE published_at IS NOT AND user_id = users.id) (Never use group by as a surrogate for IN/EXISTS. It forces the server to do a lot of unnecessary work) Is that really unintuitive? Perhaps it's just acclimation, but I find those incredible easy to grep, with the MongoDB example being a variant of the same thing. Of course that's just for very basic queries. Aggregations in MongoDB are far from intuitive (http://browsertoolkit.com/fault-tolerance.png http://browsertoolkit.com/fault-tolerance.png).
- angilly 16y ago
  I'm no SQL guru, but won't that subselect will break down when users and events grow to the millions? I probably should have thought out the example a little more. We don't ever actually write that kind of query against our production database @Punchbowl. We have a data warehouse pull out high level stats every night, and we query that. WRT aggregations, you're right -- they do require a bit of acclimation. Once you write a few, though, you're good to go.
  - ergo98 16y ago
    >I'm no SQL guru, but won't that subselect will break down when users and events grow to the millions? A moderately decent database server would have no issues in such a case, however yes, optimization would be case-specific, just as it would be with MongoDB. I was simply comparing readability of SQL for the example that you gave.
  - angilly 16y ago
    Got it. I agree. The "where exists (select 1...)" thing was neat. Learned me something.
sv123 16y ago
I like the idea of Mongo and have read about it but my relational mind can not be unwrapped. Maybe I am trying to complicate things too much. In that example if you have multiple people attending one event and you want to update the event name, do you have to go through all the people and all their events and update each one? Seems like that would be a lot of work, unless there is a way to query for that type of thing? And if there is a query, is it efficient? Seems like behind the scenes mongo would just be iterating through the users, but I'm sure there is more to it than that.
- rabidsnail 16y ago
  In real life you would have a users collection and an events collection, and the events field of users would be an array of event ids.
  - angilly 16y ago
    My first response was "not necessarily," but I'm gonna ratchet it up a notch to a flat out "you're wrong." Storing event_ids as an array in the events field defeats the whole purpose of organizing your data into rich documents. In real life, you very well could build something where the events info was built straight into the user record.
    - deleted 16y ago
      [deleted]
  - sv123 16y ago
    seems like a relational database to me...only without the ability to join?
    - kchodorow 16y ago
      Sort of. MongoDB tried to keep a lot of the trimmings of a relational database, but remain scalable. Joins are handy, but don't scale well.
  - 16y ago
rabidsnail 16y ago
Three warnings when using mongodb. None of these are enough to say not to use it, but they're things you need to watch out for: 1. Don't run javascript on a production db node. db.eval locks the node it's running on until it finishes, so the performance of that node will go down the tubes. Mapreduce is less bad in this regard because it does yield, but it does so too infrequently. If you want to use mongo's built-in javascript interpreter for anything other than development and administration, set up a slave to run your scripts on. 2. Don't use 1.6.1. If you're using 1.6.1 right now, upgrade to 1.6.2. 1.6.1 has a nasty crashing bug that had my mongo node going down about once a day and not coming up without running --repair. 3. Evaluate how much data loss costs you. Mongo stages writes in memory, and so if the db crashes hard it's likely that there will be some data that hasn't made it to disk yet. If you're building a social network the cost of some potential data loss is probably much less than the savings in hardare, admin costs, development costs, etc. But if you're a payment processor or a gambling site, stick with postgres.
- houseabsolute 16y ago
  > If you're building a social network the cost of some potential data loss is probably much less than the savings in hardare, admin costs, development costs, etc. This also depends on what is being stored. I would be unhappy if Facebook lost any of my data, and my understanding is that they use safe storage mechanisms (ones where the commit goes all the way to disk before returning) for everything except transient views like the news feed and search. Also, I don't think it's clear that MongoDB has significantly improved either admin or development costs over its safe competition, so we probably only need to look at performance wins.
- angilly 16y ago
  #3: You can force flush any write that needs to go straight to disk. insertion & update commands (at least in JS and Ruby) take an argument allowing you specify "safe_mode: true"
- mark_l_watson 16y ago
  re: potential data loss: I typically run 1 master and one read-only slave on another server. Replication of writes from the master to the read-only slave is very fast. Sure, there is a window for data loss, but for crucial data use a relational database. Also, it is great to be able to really load down the read-only slave for analytics, etc.
itgoon 16y ago
The reason you don't keep files in your database is that file systems are much better at handling files. Faster, more efficient, basically all the reasons that a single-purpose layer tends to be faster than a general-purpose layer. Databases are much better at handling discrete data than file systems - that's what they are built for. Sure, I could keep my data in a bunch of little files, but that doesn't work as well. (MS SQL has a feature where you "store" the file in the database, but the db writes the file to the filesystem, and just maintains a pointer to the actual file - not a bad hybrid) I don't know how well GridFS stacks up (it is on my todo list), although I do like the idea of replication and sharding being built in. My gut (which has been wrong before) says that it is good for websites, not so good for general storage. I use MongoDB for the same reason as mrkurt: prototyping new schemas is a breeze. I still find myself reaching for the old RDBMS toolbox as things move along, grow, and stabilize. Sometimes, a JOIN _is_ the right tool for the job.
- tcc619 16y ago
  "file systems are much better at handling files" What about batch processing a large number of small files? say 10 million image files of 500KB. A typical file system will need to seek each small file. I wonder if GridFS stores small files in blocks to allow efficient batch retrieval for processing.
  - lobster_johnson 16y ago
    GridFS is just a standard convention of how to map files to key-value stores like MongoDB -- you can implement GridFS over MongoDB in just a few lines of Ruby code. GridFS breaks files into fixed-size chunks, and uses a single MongoDB document per chunk. It's not exactly rocket science. The author of the blog post touts it as a _feature_ of MongoDB, but it's more accurate to say that it's an artifact of MongoDB's 4MB document size limit -- you simply cannot store large files in MongoDB without breaking them up. Sure, by splitting files into chunks you can parallelize loading them, but that's about the only advantage. Among the key-value NoSQL databases, Cassandra and Riak are much better at storing large chunks of data -- neither has a specific limit on the size of objects. I have used both successfully to store assets such as JPEGs, and they are both extremely fast both on reads and on writes. Neither is built for that purpose, and will load an entire object into memory instead of streaming it, so if you have lots of concurrent queries you will simply run out of memory at some point -- 10 clients each loading a 10MB image at the same time will have the database peak at 100MB at that moment. Actually, Riak uses dangerously large amounts of memory when just saving a number of large files. I don't know if that's because of Erlang's garbage collector lagging behind, or what; I would be worried about swapping or running out of memory when running it in a production system.
Semiapies 16y ago
4) You're using a 64-bit OS. http://blog.mongodb.org/post/137788967/32-bit-limitations http://blog.mongodb.org/post/137788967/32-bit-limitations
weixiyen 16y ago
4) For web apps at least, it's nice to get data back in json format that you can give to the browser immediately without having to build your own json object each time. MongoDB saves you development time at every level. Add to the fact that it is easy to horizontally scale. I'm using mongodb right now and the synergy between jquery - node.js - mongodb is simply amazing.
- angilly 16y ago
  Totally agree.
TomasSedovic 16y ago
To my (limited, I admit) knowledge of databases every one of these reasons was a reason to use CouchDB as well. When the author got to the real arguments, he kept comparing MongoDB to SQL databases and the jab at CouchDB (and the other non-relational databases) seemed without merit to me. I'm sure there are good reasons to use mongo over couch but I don't think they're the ones listed here.

35 points by angilly 16y ago

Really, the #1 reason to use MongoDB (if you're me, anyway) is to save development time associated with making your relational schema start small and change as your new app progresses. I feel a smug sense of joy every time I add a field somewhere, or delete another, or create some kind of nested document. It's taken me a while to really understand how many compromises I used to make because changing schemas is a pain in the ass. Simplified queries, though, are a knock against mongo. Joins are great and I would like to do joins on my Mongo documents, but I end up having to replicate a lot of that in code. Sure it's nice that a document can be more complex and you don't spend a lot of time moving things into tables that are really part of the same record. It's nice because it's not forced, though, not because keeping data in different tables is always the wrong way to do things.

weixiyen 16y ago
A lot sites these days don't do joins with relational databases anyways. They just use mysql or postgresql as a key value store.
saurik 16y ago
I use PostgreSQL and change my schema constantly, adding/removing columns, changing data types, switching around foreign key constraints, all within safely guarded transactions that I can roll back if I realize that I'm doing something wrong. (And yes: adding/removing/renaming columns is "instant": it doesn't actually do the work of rewriting existing rows on disk.) Frankly, I also use MongoDB, and I'm terrified of screwing much with the schema, because then I either a) have to make certain I have anal documentation about what fields are in use on what subsets of objects, keeping code around to make certain to detect and interpret old kinds of data, or b) use "simplified queries" (really, write a bunch of manual code as if I didn't have a query model at all) in order to find and update these old objects, non-atomically and with no transaction safety. Seriously: the only reason I've so far heard for why having dynamic type verification in my database server is valuable is when you have /so much data/ that it is now fundamentally infeasible to make changes to it with any centralized transaction control--specifically, Google's "it's always the interim somewhere" scenario--not because it is somehow more convenient to do so when you only have in the tens of millions of rows.

ergo98 16y ago

MongoDB is elegant and remarkably powerful. Every developer should run through the excellent getting started tutorial they have for it, as it really is eye opening. http://www.mongodb.org/display/DOCS/Tutorial http://www.mongodb.org/display/DOCS/Tutorial However I have to respectfully disagree on the "Simple queries" bit. The SQL example given is kind of terrible, however how about- SELECT * FROM users WHERE id IN (SELECT user_id FROM events WHERE published_at IS NOT NULL) or SELECT * FROM users WHERE EXISTS (SELECT 1 FROM events WHERE published_at IS NOT AND user_id = users.id) (Never use group by as a surrogate for IN/EXISTS. It forces the server to do a lot of unnecessary work) Is that really unintuitive? Perhaps it's just acclimation, but I find those incredible easy to grep, with the MongoDB example being a variant of the same thing. Of course that's just for very basic queries. Aggregations in MongoDB are far from intuitive (http://browsertoolkit.com/fault-tolerance.png http://browsertoolkit.com/fault-tolerance.png).

angilly 16y ago
I'm no SQL guru, but won't that subselect will break down when users and events grow to the millions? I probably should have thought out the example a little more. We don't ever actually write that kind of query against our production database @Punchbowl. We have a data warehouse pull out high level stats every night, and we query that. WRT aggregations, you're right -- they do require a bit of acclimation. Once you write a few, though, you're good to go.
- ergo98 16y ago
  >I'm no SQL guru, but won't that subselect will break down when users and events grow to the millions? A moderately decent database server would have no issues in such a case, however yes, optimization would be case-specific, just as it would be with MongoDB. I was simply comparing readability of SQL for the example that you gave.
- angilly 16y ago
  Got it. I agree. The "where exists (select 1...)" thing was neat. Learned me something.

sv123 16y ago

I like the idea of Mongo and have read about it but my relational mind can not be unwrapped. Maybe I am trying to complicate things too much. In that example if you have multiple people attending one event and you want to update the event name, do you have to go through all the people and all their events and update each one? Seems like that would be a lot of work, unless there is a way to query for that type of thing? And if there is a query, is it efficient? Seems like behind the scenes mongo would just be iterating through the users, but I'm sure there is more to it than that.

rabidsnail 16y ago
In real life you would have a users collection and an events collection, and the events field of users would be an array of event ids.
- angilly 16y ago
  My first response was "not necessarily," but I'm gonna ratchet it up a notch to a flat out "you're wrong." Storing event_ids as an array in the events field defeats the whole purpose of organizing your data into rich documents. In real life, you very well could build something where the events info was built straight into the user record.
  - deleted 16y ago
    [deleted]
- sv123 16y ago
  seems like a relational database to me...only without the ability to join?
  - kchodorow 16y ago
    Sort of. MongoDB tried to keep a lot of the trimmings of a relational database, but remain scalable. Joins are handy, but don't scale well.
- 16y ago

rabidsnail 16y ago

Three warnings when using mongodb. None of these are enough to say not to use it, but they're things you need to watch out for: 1. Don't run javascript on a production db node. db.eval locks the node it's running on until it finishes, so the performance of that node will go down the tubes. Mapreduce is less bad in this regard because it does yield, but it does so too infrequently. If you want to use mongo's built-in javascript interpreter for anything other than development and administration, set up a slave to run your scripts on. 2. Don't use 1.6.1. If you're using 1.6.1 right now, upgrade to 1.6.2. 1.6.1 has a nasty crashing bug that had my mongo node going down about once a day and not coming up without running --repair. 3. Evaluate how much data loss costs you. Mongo stages writes in memory, and so if the db crashes hard it's likely that there will be some data that hasn't made it to disk yet. If you're building a social network the cost of some potential data loss is probably much less than the savings in hardare, admin costs, development costs, etc. But if you're a payment processor or a gambling site, stick with postgres.

houseabsolute 16y ago
> If you're building a social network the cost of some potential data loss is probably much less than the savings in hardare, admin costs, development costs, etc. This also depends on what is being stored. I would be unhappy if Facebook lost any of my data, and my understanding is that they use safe storage mechanisms (ones where the commit goes all the way to disk before returning) for everything except transient views like the news feed and search. Also, I don't think it's clear that MongoDB has significantly improved either admin or development costs over its safe competition, so we probably only need to look at performance wins.
angilly 16y ago
#3: You can force flush any write that needs to go straight to disk. insertion & update commands (at least in JS and Ruby) take an argument allowing you specify "safe_mode: true"
mark_l_watson 16y ago
re: potential data loss: I typically run 1 master and one read-only slave on another server. Replication of writes from the master to the read-only slave is very fast. Sure, there is a window for data loss, but for crucial data use a relational database. Also, it is great to be able to really load down the read-only slave for analytics, etc.

itgoon 16y ago

The reason you don't keep files in your database is that file systems are much better at handling files. Faster, more efficient, basically all the reasons that a single-purpose layer tends to be faster than a general-purpose layer. Databases are much better at handling discrete data than file systems - that's what they are built for. Sure, I could keep my data in a bunch of little files, but that doesn't work as well. (MS SQL has a feature where you "store" the file in the database, but the db writes the file to the filesystem, and just maintains a pointer to the actual file - not a bad hybrid) I don't know how well GridFS stacks up (it is on my todo list), although I do like the idea of replication and sharding being built in. My gut (which has been wrong before) says that it is good for websites, not so good for general storage. I use MongoDB for the same reason as mrkurt: prototyping new schemas is a breeze. I still find myself reaching for the old RDBMS toolbox as things move along, grow, and stabilize. Sometimes, a JOIN _is_ the right tool for the job.

tcc619 16y ago
"file systems are much better at handling files" What about batch processing a large number of small files? say 10 million image files of 500KB. A typical file system will need to seek each small file. I wonder if GridFS stores small files in blocks to allow efficient batch retrieval for processing.
- lobster_johnson 16y ago
  GridFS is just a standard convention of how to map files to key-value stores like MongoDB -- you can implement GridFS over MongoDB in just a few lines of Ruby code. GridFS breaks files into fixed-size chunks, and uses a single MongoDB document per chunk. It's not exactly rocket science. The author of the blog post touts it as a _feature_ of MongoDB, but it's more accurate to say that it's an artifact of MongoDB's 4MB document size limit -- you simply cannot store large files in MongoDB without breaking them up. Sure, by splitting files into chunks you can parallelize loading them, but that's about the only advantage. Among the key-value NoSQL databases, Cassandra and Riak are much better at storing large chunks of data -- neither has a specific limit on the size of objects. I have used both successfully to store assets such as JPEGs, and they are both extremely fast both on reads and on writes. Neither is built for that purpose, and will load an entire object into memory instead of streaming it, so if you have lots of concurrent queries you will simply run out of memory at some point -- 10 clients each loading a 10MB image at the same time will have the database peak at 100MB at that moment. Actually, Riak uses dangerously large amounts of memory when just saving a number of large files. I don't know if that's because of Erlang's garbage collector lagging behind, or what; I would be worried about swapping or running out of memory when running it in a production system.

Semiapies 16y ago

4) You're using a 64-bit OS. http://blog.mongodb.org/post/137788967/32-bit-limitations http://blog.mongodb.org/post/137788967/32-bit-limitations

weixiyen 16y ago

4) For web apps at least, it's nice to get data back in json format that you can give to the browser immediately without having to build your own json object each time. MongoDB saves you development time at every level. Add to the fact that it is easy to horizontally scale. I'm using mongodb right now and the synergy between jquery - node.js - mongodb is simply amazing.

angilly 16y ago
Totally agree.

TomasSedovic 16y ago

To my (limited, I admit) knowledge of databases every one of these reasons was a reason to use CouchDB as well. When the author got to the real arguments, he kept comparing MongoDB to SQL databases and the jab at CouchDB (and the other non-relational databases) seemed without merit to me. I'm sure there are good reasons to use mongo over couch but I don't think they're the ones listed here.