3 ms·

Lack of isolation and multi document atomic updates is one of the biggest difficulties. If you are updating multiple associated documents under a semantic trans

Lack of isolation and multi document atomic updates is one of the biggest difficulties. If you are updating multiple associated documents under a semantic transaction, it's possible there will be a window in which the partial update is visible. Sometimes this is harmless, other times quite harmful (changing constraints and budget on a bidding platform can result in overpaying for undesired inventory for example). This can be worked around somewhat. If you only ever update the parent documents, then you can create new associated documents and atomically update the parent document to point to all the new ones. It's a bit of a song and dance, and definitely has limitations and gets complicated fast.

gwbas1c 10y ago
I'm painfully aware of the "Lack of isolation and multi document atomic updates is one of the biggest difficulties" problem, as I encountered it when I tried working with MongoDB. (I chose MongoDB because I was working with very fluid requirements and needed a very flexible schema.) But that's not really what the article complains about! The context is that I met some of the Diaspora leads in the summer of 2010. At the time, they were ambitious, but very inexperienced. (They did teach me some valuable lessons about encryption!) The reason why I say this, is without some kind of data, it's hard to know if "don't emulate joins in MongoDB" is a conclusion of inexperience; and something that a more experienced developer would understand how to do correctly. For example, in a message board application, if a join is emulated between a discussion and user objects that just have username and avatar; the penalty of seeing an incomplete update is inconsequential. Either the user sees the old name / avatar, or the user sees the new name / avatar. So the incomplete update problem (in theory) can be handled by restricting "joins" to only when it's okay to see old data. So, assuming that the incomplete update problem is solvable, at what point do emulated joins stop scaling?
- phamilton 10y ago
  > by restricting "joins" to only when it's okay to see old data. So what do you do when it's not OK to see old data? That's when it breaks down. There are definitely ways around the issues, but you have to make compromises in how you build the application. If you don't make those design decisions early, you find yourself doing it wrong. In describing large scale systems, a friend of mine said "Imagine a scenario that has a one in a billion chance of happening. On a 10k qps system, it happens almost daily." That's how this breaks down at scale. Awkward corner cases that don't really seem likely suddenly become daily events.
  - gwbas1c 10y ago
    So, would you say that it's a solvable problem, but it's so time consuming or difficult that a traditional relational database is a more appropriate database for average developers? (Remember, the Diaspora guys were right out of college and inexperienced.) It's as if MongoDB is great for prototyping, and great for lossy data at scale, but it can't hit the middle.
    - phamilton 10y ago
      Mongo's problem isn't really the lack of schema or joins or anything. It's the mountain of false assumptions people make around its behavior. If you understand the limitations of the libraries and systems you use, you can solve most problems. The issue is that most people don't understand those limitations. Mongo gets an especially bad rap because it actively pushed misleading information as part of its sales pitch.

gwbas1c 10y ago

I'm painfully aware of the "Lack of isolation and multi document atomic updates is one of the biggest difficulties" problem, as I encountered it when I tried working with MongoDB. (I chose MongoDB because I was working with very fluid requirements and needed a very flexible schema.) But that's not really what the article complains about! The context is that I met some of the Diaspora leads in the summer of 2010. At the time, they were ambitious, but very inexperienced. (They did teach me some valuable lessons about encryption!) The reason why I say this, is without some kind of data, it's hard to know if "don't emulate joins in MongoDB" is a conclusion of inexperience; and something that a more experienced developer would understand how to do correctly. For example, in a message board application, if a join is emulated between a discussion and user objects that just have username and avatar; the penalty of seeing an incomplete update is inconsequential. Either the user sees the old name / avatar, or the user sees the new name / avatar. So the incomplete update problem (in theory) can be handled by restricting "joins" to only when it's okay to see old data. So, assuming that the incomplete update problem is solvable, at what point do emulated joins stop scaling?

phamilton 10y ago
> by restricting "joins" to only when it's okay to see old data. So what do you do when it's not OK to see old data? That's when it breaks down. There are definitely ways around the issues, but you have to make compromises in how you build the application. If you don't make those design decisions early, you find yourself doing it wrong. In describing large scale systems, a friend of mine said "Imagine a scenario that has a one in a billion chance of happening. On a 10k qps system, it happens almost daily." That's how this breaks down at scale. Awkward corner cases that don't really seem likely suddenly become daily events.
- gwbas1c 10y ago
  So, would you say that it's a solvable problem, but it's so time consuming or difficult that a traditional relational database is a more appropriate database for average developers? (Remember, the Diaspora guys were right out of college and inexperienced.) It's as if MongoDB is great for prototyping, and great for lossy data at scale, but it can't hit the middle.
  - phamilton 10y ago
    Mongo's problem isn't really the lack of schema or joins or anything. It's the mountain of false assumptions people make around its behavior. If you understand the limitations of the libraries and systems you use, you can solve most problems. The issue is that most people don't understand those limitations. Mongo gets an especially bad rap because it actively pushed misleading information as part of its sales pitch.