5 ms·

My position on mongo is thus: Its goal is humongous data sets, hence the name. Until well proven, I'm not the type to use it for huge data, but will keep an ey

My position on mongo is thus: Its goal is humongous data sets, hence the name. Until well proven, I'm not the type to use it for huge data, but will keep an eye out for case studies. I have used Mongo on two projects with reasonably small data sets. My largest collection at the moment is 5 million, and that's basically a log. Other collections are less than 100,000. I've been running mongo 1.6 for a year on these two sites without so much as a hiccup. I do the normal very simple things to protect myself: a cron job to dump db and then copy it to a backup server. And that's it. I enjoy using mongo for these projects because when I want to add a new feature to one of my domain models, I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving. I think the mongo folks are fantastic in their open dev process and maybe one day, some threshold will be crossed where I can say that for certain types of big data usage mongo is a clear solid choice.

ehthere 15y ago
'I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.' That's exactly the same as adding a new column to your DB with NULL as the default value.
- philjackson 15y ago
  Not really. Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience). Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.
  - ehthere 15y ago
    You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'. The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.
    - philjackson 15y ago
      "You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value?" Taking the team I worked with at the BBC as an example: 1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this happened we would have to drop the database back to a known, good state. You can't have columns with constraints left around when the code which might have satisfied those constraints is reverted. Doing it without migrations would have been idiotic to say the least. 2) Developers work on different features in different branches, often collaborating. Different features apply new attributes to the db schema. It's important for a developer to know his DB is in the correct state when he starts hacking. You do that with migrations. Because you almost completely remove the need for schema definition (and what little of it you do, you can do in app. code) you simply don't need the migrations any more. Using mongo means you can pretty much just export your applications domain without having to coerce it into the relational model. "I don't see what you gain by having both 'undefined' and 'null'." They mean totally different things. Undefined means that the field has never been explicitly set, null means the field has been set. This means you know what's been backfilled and what hasn't - you can't tell without extra metadata in mysql. Also, in mysql if you provide a null default then every row has to be updated. "It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea." I don't know BASIC but you can put Perl into a certain configuration that allows this. That makes for horrible scoping issues that aren't analogous or applicable to what we're talking about.
      - ehthere 15y ago
dextorious 15y ago
"""My position on mongo is thus: Its goal is humongous data sets, hence the name. Until well proven, I'm not the type to use it for huge data, but will keep an eye out for case studies.""" Actually Mongo is bad for really humongous data sets. It works well if the working data set (the data you commonly need) can fit in memory. Of course this doesn't scale very well with say several terabytes of data, while there are Oracle databases that handle a lot more... In the case you Mongo you go to sharding etc and things get complicated in your app handling.
- philjackson 15y ago
  "In the case you Mongo you go to sharding etc and things get complicated in your app handling." Can you elaborate on this?

ehthere 15y ago

'I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.' That's exactly the same as adding a new column to your DB with NULL as the default value.

philjackson 15y ago
Not really. Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience). Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.
- ehthere 15y ago
  You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value? I don't see what you gain by having both 'undefined' and 'null'. The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.
  - philjackson 15y ago
    "You don't need to do any 'migration thing', you just add the column to the DB and choose a sensible default value?" Taking the team I worked with at the BBC as an example: 1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this happened we would have to drop the database back to a known, good state. You can't have columns with constraints left around when the code which might have satisfied those constraints is reverted. Doing it without migrations would have been idiotic to say the least. 2) Developers work on different features in different branches, often collaborating. Different features apply new attributes to the db schema. It's important for a developer to know his DB is in the correct state when he starts hacking. You do that with migrations. Because you almost completely remove the need for schema definition (and what little of it you do, you can do in app. code) you simply don't need the migrations any more. Using mongo means you can pretty much just export your applications domain without having to coerce it into the relational model. "I don't see what you gain by having both 'undefined' and 'null'." They mean totally different things. Undefined means that the field has never been explicitly set, null means the field has been set. This means you know what's been backfilled and what hasn't - you can't tell without extra metadata in mysql. Also, in mysql if you provide a null default then every row has to be updated. "It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea." I don't know BASIC but you can put Perl into a certain configuration that allows this. That makes for horrible scoping issues that aren't analogous or applicable to what we're talking about.
    - ehthere 15y ago

dextorious 15y ago

"""My position on mongo is thus: Its goal is humongous data sets, hence the name. Until well proven, I'm not the type to use it for huge data, but will keep an eye out for case studies.""" Actually Mongo is bad for really humongous data sets. It works well if the working data set (the data you commonly need) can fit in memory. Of course this doesn't scale very well with say several terabytes of data, while there are Oracle databases that handle a lot more... In the case you Mongo you go to sharding etc and things get complicated in your app handling.

philjackson 15y ago
"In the case you Mongo you go to sharding etc and things get complicated in your app handling." Can you elaborate on this?