7 ms·

I posted this further down the thread, but I thought I'd share my thoughts on why I like mongo. Most people don't like mongo because 10gen gives the impression

by bkanber 13y ago

I posted this further down the thread, but I thought I'd share my thoughts on why I like mongo. Most people don't like mongo because 10gen gives the impression that mongo is better than it actually is, many people feel that mongo is not reliable enough for at-scale applications. They're right; it's not. But that's ok, because: Mongo's really great for rapid prototyping. You don't need to worry about updating the schema at the db level, it can store any type of document in any collection without complaining, it's really easy to install and configure, the query language is simple and only takes a couple of minutes to learn, it's pretty fast in most use cases, it's pretty safe in most use cases, and it's easy to create a replica set once your prototype gets usage and starts scaling. Mongo does everything well up until you reach the level where you need heavy-hitting, at-scale, mission-critical performance and reliability. Most projects out there (99 in 100?) will never reach the level of scale that requires better tools than mongo. And since the rest of it is so easy to use, that makes mongo a great starting point for most projects. You can always switch databases later, but mongo gives you the flexibility to concentrate on more important things in the early stages of a project.

swanson 13y ago
Couldn't you argue that e.g. Postgres and ActiveRecord give you the same rapid prototyping ability but with an easier (and more established) path towards scalability? It is easy to change your schema with migrations at the beginning of a project - just go edit the original ones and nuke your database. And I don't have to worry about properly configuring write-locks, replica sets, or writing map reduce javascript.
- threeseed 13y ago
  Configuring write-locks ? I guess you have never actually used MongoDB before which explains why absolutely none of what you said makes any sense. MongoDB is far easier to use, manage the schema with and scale than PostgreSQL.
  - mritun 13y ago
    ... if you don't care about your data! It's not a database that by default just throws your data at the wall and well, doesn't even care to look if it stuck.
- bkanber 13y ago
  Of course you could argue that. But so what? Having an easier path towards scalability is nice, but irrelevant for the vast majority of projects; not every project is going to turn into a startup or a real product or even something you work on for more than a few weekends! The last time you hacked together a blogging engine in Node.js one weekend, were you worried about future scalability, or just playing with new technologies because it's fun? And while it's pretty easy to do schema migrations, it's not easier than _not_ doing them. And what is it you really want to worry about? Making sure your DB is production ready, or tinkering with Express.js and Backbone? So because of that, many people use mongo as their de facto database. It's just what I use when I need a persistence layer for anything I build, because I already have a mongo db running for like 30 different defunct projects on my dev server. And then, by happy accident, one of your side projects turns into a real product, and then mongo handles you really well for the first year or so, just up til the point of having to hire a real devops engineer; at which point you swap out your ORM layer and switch to postgres.
  - ollysb 13y ago
    > while it's pretty easy to do schema migrations, it's not easier than _not_ doing them. I just don't buy this argument, writing and executing migrations is braindead simple and usually takes what, 20 seconds start to finish? Writing the line of code you need for mongo must be about 5 seconds. edit: I did actually give mongodb a good crack(used on a side-project for 6 months last year) but I found that I actually spent a huge proportion of my time working around things that were missing compared to ActiveRecord. It was a huge net loss for me in terms of productivity.
  - empthought 13y ago
    > postgre. Just so you know, that's not actually a thing.
    - dkural 13y ago
      He meant postgres, as you also know. When autocorrect fails, there is empthought.
  - 13y ago
- rapind 13y ago
  Actually postgres can be a bit of a PITA, but so can mongo. At the risk of sounding reckless, unless the app needs to support high CUD throughput I sometimes opt for sqlite. Doesn't get much easier than that and it's read performance is impressive from what I've seen. Even then you can sometimes get away with staying on sqlite for your admin side CRUD and redis for the heavy / concurrent writing from the public facing side (obviously situational).
  - bkanber 13y ago
    I've been using sqlite more and more as well. Super lightweight, but since it's SQL most ORMs can handle switching to MySQL/postgres really easily if you ever need to make the switch.
    - chris_wot 13y ago
      Now this is an important point. Once you make your product, what do you need to do to retool a major part of the application? This seems like an excellent approach. Many people in the Java world use something very simple like hsqldb, then shift to a new database when out of development.
danpalmer 13y ago
The case for using it as a prototyping database is the best use-case I've seen for Mongo, however I'm not sure it's always a good idea. For a hack-weekend sort of project, fine, but if you are in any way attempting to make a product, it strikes me as the sort of thing that would be really difficult to change later down the line, and so worth investing the very little extra effort it takes to include your schema in the database, and use something like Postgres/MySQL/etc.
- threeseed 13y ago
  I've never understand people that worry so much about the schema. It's like they've missed the last decade of computing. Everybody these days uses ORM. Which means that (a) data migration between databases is a relatively simple task and (b) schemas often just get in your way.
- bkanber 13y ago
  I feel the opposite way. If your application's data layer is sensibly designed, it shouldn't be too bad to switch to postgres when you need to. It may be tedious if you have a large codebase, but it won't be difficult. I think the up-front benefits of using mongo (especially as a sole developer/devops/sysadmin person) outweigh the difficulty of the changes you'll need to make later on, which will only happen as you hit scale and have more resources to nurture the devops side of the tech stack.
  - dboat 13y ago
    This prototyping story sounds to me like admittedly shooting yourself in the foot if your prototype turns out to be worth a damn. Basically, you're saying mongo is an excellent choice only when your storage backend is a moot point. I can't think of any codebases I've seen where intentionally choosing the storage backend you know you don't want to use (if the project is successful) would be a reasonable thing to do. Understate it if you must, but having to change your backend from mongo to postgres is not a desirable situation. Besides, if you're going to use postgres to scale, use it's features and write well optimized queries for it. The difference between a bad massively complex query and a well optimized one can be several orders of magnitude, and that optimization can indeed be difficult. It goes without saying that you wouldn't leave that to an ORM.
lebski88 13y ago
Mongo is also brilliant for internal tools that need to change rapidly but you absolutely know will never require significant scale. I've been badly burnt trying to use mongo on a large dataset but it's genuinely great for getting things done quickly. I also used it to write a service that had to go from nothing to working in a couple of days. I then spent the next two days swapping it back out again. It was surprisingly painless to go from an object store to a relational model
bcoates 13y ago
I just use the filesystem for that sort of thing. Everyone justifies using MongoDB because its easy and general but compared to the tool and compatibility ecosystem around files it's awkward and primitive.
- functional_test 13y ago
  The file system is great for some things, but, for example, how would you handle the same data to having multiple indices?
  - barrkel 13y ago
    Links - both soft and hard.
    - functional_test 13y ago
      Wouldn't that become quite a bit of book keeping though?
      - barrkel 13y ago
        Maintaining an index is a cost of the lookup efficiencies gained.
JulianMorrison 13y ago
Have you considered RethinkDB? For most of the advantages of MongoDB that don't specifically come from mmap and overwrite-in-place, it's an equal or better.
- ahoge 13y ago
  RethinkDB still doesn't support Windows. [1] Most developers who use Windows will just go with something which doesn't require a virtual machine. For example, MongoDB, CouchDB, OrientDB, Cassandra, and ArrangoDB work fine everywhere. [1] https://github.com/rethinkdb/rethinkdb/issues/1100 https://github.com/rethinkdb/rethinkdb/issues/1100
  - dodyg 13y ago
    Did you try RavenDB which works quite well in Windows?
    - ahoge 13y ago
      I'm only interested in cross platform stuff. Whichever part of the stack it is, it must work on Windows, Mac, and Linux.
ozataman 13y ago
Application design for me almost always begins with data and data structures. Whether my database has an explicit schema or not, I always have one in mind, documented or otherwise reified in the table-data structures I have in my code. I just don't get why people would want a schema-free database that is in almost every way inferior to the rock-solid power beast that is Postgres. Just use a library with proper migration support so you can propagate changes to your schema rapidly during development. You'll thank us later when you learn a little bit of SQL and start analyzing your data, running circles around the no-sql guys. Cassandra et. al. are completely different, in that you don't use them because they are more fun to use. You use them despite their awkward, low-level interfaces because you're going to dump billions of data cells into your database from day one with no end in sight and want all the easy scaling/availability features provided.
- bkanber 13y ago
  Do you always start with the perfect data structure? I find myself adding, removing, and restructuring schema often. Just as you think it's silly to use an "inferior" db during prototyping, I think it's silly to have to jump through hoops -- even minor ones -- while I'm just trying to experiment with a new technology or play with a concept, product design, or pet project. 99 times out of 100, I don't care if my project survives the weekend. Let me use that database I want to use! > You'll thank us later when you learn a little bit of SQL and start analyzing your data, running circles around the no-sql guys. That's a little condescending... do you know a single mongo user who doesn't have experience with SQL? Plus, I love the fact that I can literally run javascript against my database. Good for production? Certainly not. But that doesn't mean it's not fun or useful. Not every project requires such rigor. If that's how you enjoy development, that's great! Very few of my projects put the db layer to the test, and so I'm happy with the balance that mongo gives me. I use it in about 4/5 of my experiments and side projects.
  - integraton 13y ago
    > Do you always start with the perfect data structure? I find myself adding, removing, and restructuring schema often. Which is why it doesn't make any sense to claim that using MongoDB somehow eliminates needing to migrate your data as it evolves.
rsynnott 13y ago
> Mongo's really great for rapid prototyping. Has any phrase ever struck more fear into the heart of a programmer? Rapid prototypes have a nasty tendency of accidentally becoming products... > You can always switch databases later shudder
- nostrademons 13y ago
  That's why you use a language or technology that's politically unfeasible for your rapid prototypes, like Clojure or Haskell, or...for that matter...MongoDB. ;-)
  - chris_wot 13y ago
    In certain companies, MongoDB is the new hotness. This could very much backfire on you!
  - twic 13y ago
    Someone tried that at my company. We now have a Clojure app in production. Brilliant.
    - nostrademons 13y ago
      Hey, it's a great way to get lesser-known languages into production! </slight-sarcasm> I wonder if I'm going to be the one who finally switches Google Search over to Go, by way of a quick throwaway prototype...
EGreg 13y ago
I think they are upset with their marketing, like "web scale". Even the name "Mongo" is derived from "Humongous" -- but that's exactly the scale at which you'd switch away from Mongo. Reminds me of this: http://www.youtube.com/watch?v=URJeuxI7kHo http://www.youtube.com/watch?v=URJeuxI7kHo Want NoSQL? Use RIAK!
- rsynnott 13y ago
  > Want NoSQL? Use RIAK! See, you're just perpetuating The NoSQL Problem. :) Riak is well-suited to some tasks, but it is no more a magical fits-every-problem thing than MongoDB is.
  - EGreg 13y ago
    I sometimes throw out these things as a quick way to get reactions and interesting feedback as to why something is good/bad. For instance right now I know mostly good things about Riak, that's why I posted this ending to my comment.
    - dboat 13y ago
      Having recently rolled out riak into a production environment I can offer some off-the-cuff bullet points: - Mostly easy to work with. Mapreduces can be a big pain to troubleshoot because you can't console.log() in your JS. Didn't try it in erlang. - Being masterless, it has a very good replication story for servers _in the same data center_. It really bit us that there was no good riak solution for syncing data across multiple data centers. There is an enterprise solution for that, but it's quite expensive, which makes riak less appealing if you don't have much budget on your project. - Errors in general are next to useless. Get comfortable waiting for answers in IRC when you get opaque error messages after running queries. You can definitely work past this, but it wasted a lot of my time. - Not sure if pro or con, but as the cluster reached load capacity, from a combination of data size and read requests, map functions would begin to slowly fail. After a while, we could tell which completely useless error message (preflist_exhausted, my old friend) could be fixed by a cluster restart, and which would simply begin to happen with greater frequency as more data was added. This was exacerbated by my company refusing to pay for anything more than a three node cluster. You might say I should have fought harder for more, but I had to fight to make them not host all three nodes on a single server. There are places that will hire you that simply do not intend to do anything sane, but I digress. The takeaway: riak is not a super cheap way to scale. - Bulk inserts? What are bulk inserts? - Key filtering is just a shim over listing all keys in a bucket. Further, listing all keys in a bucket, or all buckets in a cluster, can be very expensive, and basically you'd never do it unless you had a very small bucket. The bag of tricks you can apply to speed up slow queries is basically "Do you have secondary indexes? Ok, good." Those points do read a little negative, but actually I would use riak again. To me it works best as a temporary event store living in one data center. If you've got a bunch of items shuffling around your backend in real time, being processed to and fro, you could definitely do worse than sticking in it riak and adding more nodes as needed.
lucian1900 13y ago
None of the things you listed are any harder with Postgres and SQLAlchemy. Learning to use MongoDB isn't exactly trivial anyway, so why choose the thing that is known to be broken at all, when it's neither easier nor faster?

by bkanber 13y ago

swanson 13y ago

Couldn't you argue that e.g. Postgres and ActiveRecord give you the same rapid prototyping ability but with an easier (and more established) path towards scalability? It is easy to change your schema with migrations at the beginning of a project - just go edit the original ones and nuke your database. And I don't have to worry about properly configuring write-locks, replica sets, or writing map reduce javascript.

threeseed 13y ago
Configuring write-locks ? I guess you have never actually used MongoDB before which explains why absolutely none of what you said makes any sense. MongoDB is far easier to use, manage the schema with and scale than PostgreSQL.
- mritun 13y ago
  ... if you don't care about your data! It's not a database that by default just throws your data at the wall and well, doesn't even care to look if it stuck.
bkanber 13y ago
Of course you could argue that. But so what? Having an easier path towards scalability is nice, but irrelevant for the vast majority of projects; not every project is going to turn into a startup or a real product or even something you work on for more than a few weekends! The last time you hacked together a blogging engine in Node.js one weekend, were you worried about future scalability, or just playing with new technologies because it's fun? And while it's pretty easy to do schema migrations, it's not easier than _not_ doing them. And what is it you really want to worry about? Making sure your DB is production ready, or tinkering with Express.js and Backbone? So because of that, many people use mongo as their de facto database. It's just what I use when I need a persistence layer for anything I build, because I already have a mongo db running for like 30 different defunct projects on my dev server. And then, by happy accident, one of your side projects turns into a real product, and then mongo handles you really well for the first year or so, just up til the point of having to hire a real devops engineer; at which point you swap out your ORM layer and switch to postgres.
- ollysb 13y ago
  > while it's pretty easy to do schema migrations, it's not easier than _not_ doing them. I just don't buy this argument, writing and executing migrations is braindead simple and usually takes what, 20 seconds start to finish? Writing the line of code you need for mongo must be about 5 seconds. edit: I did actually give mongodb a good crack(used on a side-project for 6 months last year) but I found that I actually spent a huge proportion of my time working around things that were missing compared to ActiveRecord. It was a huge net loss for me in terms of productivity.
- empthought 13y ago
  > postgre. Just so you know, that's not actually a thing.
  - dkural 13y ago
    He meant postgres, as you also know. When autocorrect fails, there is empthought.
- 13y ago
rapind 13y ago
Actually postgres can be a bit of a PITA, but so can mongo. At the risk of sounding reckless, unless the app needs to support high CUD throughput I sometimes opt for sqlite. Doesn't get much easier than that and it's read performance is impressive from what I've seen. Even then you can sometimes get away with staying on sqlite for your admin side CRUD and redis for the heavy / concurrent writing from the public facing side (obviously situational).
- bkanber 13y ago
  I've been using sqlite more and more as well. Super lightweight, but since it's SQL most ORMs can handle switching to MySQL/postgres really easily if you ever need to make the switch.
  - chris_wot 13y ago
    Now this is an important point. Once you make your product, what do you need to do to retool a major part of the application? This seems like an excellent approach. Many people in the Java world use something very simple like hsqldb, then shift to a new database when out of development.

danpalmer 13y ago

The case for using it as a prototyping database is the best use-case I've seen for Mongo, however I'm not sure it's always a good idea. For a hack-weekend sort of project, fine, but if you are in any way attempting to make a product, it strikes me as the sort of thing that would be really difficult to change later down the line, and so worth investing the very little extra effort it takes to include your schema in the database, and use something like Postgres/MySQL/etc.

threeseed 13y ago
I've never understand people that worry so much about the schema. It's like they've missed the last decade of computing. Everybody these days uses ORM. Which means that (a) data migration between databases is a relatively simple task and (b) schemas often just get in your way.
bkanber 13y ago
I feel the opposite way. If your application's data layer is sensibly designed, it shouldn't be too bad to switch to postgres when you need to. It may be tedious if you have a large codebase, but it won't be difficult. I think the up-front benefits of using mongo (especially as a sole developer/devops/sysadmin person) outweigh the difficulty of the changes you'll need to make later on, which will only happen as you hit scale and have more resources to nurture the devops side of the tech stack.
- dboat 13y ago
  This prototyping story sounds to me like admittedly shooting yourself in the foot if your prototype turns out to be worth a damn. Basically, you're saying mongo is an excellent choice only when your storage backend is a moot point. I can't think of any codebases I've seen where intentionally choosing the storage backend you know you don't want to use (if the project is successful) would be a reasonable thing to do. Understate it if you must, but having to change your backend from mongo to postgres is not a desirable situation. Besides, if you're going to use postgres to scale, use it's features and write well optimized queries for it. The difference between a bad massively complex query and a well optimized one can be several orders of magnitude, and that optimization can indeed be difficult. It goes without saying that you wouldn't leave that to an ORM.

lebski88 13y ago

Mongo is also brilliant for internal tools that need to change rapidly but you absolutely know will never require significant scale. I've been badly burnt trying to use mongo on a large dataset but it's genuinely great for getting things done quickly. I also used it to write a service that had to go from nothing to working in a couple of days. I then spent the next two days swapping it back out again. It was surprisingly painless to go from an object store to a relational model

bcoates 13y ago

I just use the filesystem for that sort of thing. Everyone justifies using MongoDB because its easy and general but compared to the tool and compatibility ecosystem around files it's awkward and primitive.

functional_test 13y ago
The file system is great for some things, but, for example, how would you handle the same data to having multiple indices?
- barrkel 13y ago
  Links - both soft and hard.
  - functional_test 13y ago
    Wouldn't that become quite a bit of book keeping though?
    - barrkel 13y ago
      Maintaining an index is a cost of the lookup efficiencies gained.

JulianMorrison 13y ago

Have you considered RethinkDB? For most of the advantages of MongoDB that don't specifically come from mmap and overwrite-in-place, it's an equal or better.

ahoge 13y ago
RethinkDB still doesn't support Windows. [1] Most developers who use Windows will just go with something which doesn't require a virtual machine. For example, MongoDB, CouchDB, OrientDB, Cassandra, and ArrangoDB work fine everywhere. [1] https://github.com/rethinkdb/rethinkdb/issues/1100 https://github.com/rethinkdb/rethinkdb/issues/1100
- dodyg 13y ago
  Did you try RavenDB which works quite well in Windows?
  - ahoge 13y ago
    I'm only interested in cross platform stuff. Whichever part of the stack it is, it must work on Windows, Mac, and Linux.

ozataman 13y ago

Application design for me almost always begins with data and data structures. Whether my database has an explicit schema or not, I always have one in mind, documented or otherwise reified in the table-data structures I have in my code. I just don't get why people would want a schema-free database that is in almost every way inferior to the rock-solid power beast that is Postgres. Just use a library with proper migration support so you can propagate changes to your schema rapidly during development. You'll thank us later when you learn a little bit of SQL and start analyzing your data, running circles around the no-sql guys. Cassandra et. al. are completely different, in that you don't use them because they are more fun to use. You use them despite their awkward, low-level interfaces because you're going to dump billions of data cells into your database from day one with no end in sight and want all the easy scaling/availability features provided.

bkanber 13y ago
Do you always start with the perfect data structure? I find myself adding, removing, and restructuring schema often. Just as you think it's silly to use an "inferior" db during prototyping, I think it's silly to have to jump through hoops -- even minor ones -- while I'm just trying to experiment with a new technology or play with a concept, product design, or pet project. 99 times out of 100, I don't care if my project survives the weekend. Let me use that database I want to use! > You'll thank us later when you learn a little bit of SQL and start analyzing your data, running circles around the no-sql guys. That's a little condescending... do you know a single mongo user who doesn't have experience with SQL? Plus, I love the fact that I can literally run javascript against my database. Good for production? Certainly not. But that doesn't mean it's not fun or useful. Not every project requires such rigor. If that's how you enjoy development, that's great! Very few of my projects put the db layer to the test, and so I'm happy with the balance that mongo gives me. I use it in about 4/5 of my experiments and side projects.
- integraton 13y ago
  > Do you always start with the perfect data structure? I find myself adding, removing, and restructuring schema often. Which is why it doesn't make any sense to claim that using MongoDB somehow eliminates needing to migrate your data as it evolves.

rsynnott 13y ago

> Mongo's really great for rapid prototyping. Has any phrase ever struck more fear into the heart of a programmer? Rapid prototypes have a nasty tendency of accidentally becoming products... > You can always switch databases later shudder

nostrademons 13y ago
That's why you use a language or technology that's politically unfeasible for your rapid prototypes, like Clojure or Haskell, or...for that matter...MongoDB. ;-)
- chris_wot 13y ago
  In certain companies, MongoDB is the new hotness. This could very much backfire on you!
- twic 13y ago
  Someone tried that at my company. We now have a Clojure app in production. Brilliant.
  - nostrademons 13y ago
    Hey, it's a great way to get lesser-known languages into production! </slight-sarcasm> I wonder if I'm going to be the one who finally switches Google Search over to Go, by way of a quick throwaway prototype...

EGreg 13y ago

I think they are upset with their marketing, like "web scale". Even the name "Mongo" is derived from "Humongous" -- but that's exactly the scale at which you'd switch away from Mongo. Reminds me of this: http://www.youtube.com/watch?v=URJeuxI7kHo http://www.youtube.com/watch?v=URJeuxI7kHo Want NoSQL? Use RIAK!

rsynnott 13y ago
> Want NoSQL? Use RIAK! See, you're just perpetuating The NoSQL Problem. :) Riak is well-suited to some tasks, but it is no more a magical fits-every-problem thing than MongoDB is.
- EGreg 13y ago
  I sometimes throw out these things as a quick way to get reactions and interesting feedback as to why something is good/bad. For instance right now I know mostly good things about Riak, that's why I posted this ending to my comment.
  - dboat 13y ago
    Having recently rolled out riak into a production environment I can offer some off-the-cuff bullet points: - Mostly easy to work with. Mapreduces can be a big pain to troubleshoot because you can't console.log() in your JS. Didn't try it in erlang. - Being masterless, it has a very good replication story for servers _in the same data center_. It really bit us that there was no good riak solution for syncing data across multiple data centers. There is an enterprise solution for that, but it's quite expensive, which makes riak less appealing if you don't have much budget on your project. - Errors in general are next to useless. Get comfortable waiting for answers in IRC when you get opaque error messages after running queries. You can definitely work past this, but it wasted a lot of my time. - Not sure if pro or con, but as the cluster reached load capacity, from a combination of data size and read requests, map functions would begin to slowly fail. After a while, we could tell which completely useless error message (preflist_exhausted, my old friend) could be fixed by a cluster restart, and which would simply begin to happen with greater frequency as more data was added. This was exacerbated by my company refusing to pay for anything more than a three node cluster. You might say I should have fought harder for more, but I had to fight to make them not host all three nodes on a single server. There are places that will hire you that simply do not intend to do anything sane, but I digress. The takeaway: riak is not a super cheap way to scale. - Bulk inserts? What are bulk inserts? - Key filtering is just a shim over listing all keys in a bucket. Further, listing all keys in a bucket, or all buckets in a cluster, can be very expensive, and basically you'd never do it unless you had a very small bucket. The bag of tricks you can apply to speed up slow queries is basically "Do you have secondary indexes? Ok, good." Those points do read a little negative, but actually I would use riak again. To me it works best as a temporary event store living in one data center. If you've got a bunch of items shuffling around your backend in real time, being processed to and fro, you could definitely do worse than sticking in it riak and adding more nodes as needed.

lucian1900 13y ago

None of the things you listed are any harder with Postgres and SQLAlchemy. Learning to use MongoDB isn't exactly trivial anyway, so why choose the thing that is known to be broken at all, when it's neither easier nor faster?