4 ms·
It takes that article a long time to get to the point. What I'd like to know is, "why is emulating joins bad?" Specifically, why is it bad to load a MongoDB do
by gwbas1c 10y ago
It takes that article a long time to get to the point. What I'd like to know is, "why is emulating joins bad?"
Specifically, why is it bad to load a MongoDB document, and then load downstream documents that it links to? What kind of problems does this lead to?
Granted, when I worked with MongoDB, I encountered problems due to its lack of transactions. (It's surprisingly unreliable if you end up needing to update multiple documents; when a relational database can do that easily in a transaction.)
But, assuming you can pick an application design that does single-document updates; why is following links in a document bad?
- codyb 10y agoCan you do it at the database level? Since MDB doesn't have joins wouldn't emulation require a lot more transactions across the wire? And in a distributed system couldn't that add up very quickly?
- gwbas1c 10y agoI'm looking for a more quantitative answer. For example, if I write a message board in MDB: - An entire discussion (think hacker news story and all corresponding discussion) is a single document - Each post, reply, ect, has a user ID - I load the post object and then do a "where uid in (all uids in document)" query Obviously, that has scalability limits, but what are they? In practice, when do you hit them?
- codyb 10y agoSounds like a great article. Take some of the open sourced reddit comments, or just create a small script to generate lorem ipsum posts for "users". Deploy mongo somewhere. Scale the data and scale the number of requests per second and see when things fall off. I've got so many projects I'd like to do but that does sound neat. I'd like to compare it to couchbase and postgres as well, two databases I'm much more interested in, and also Neo4j which I'm sure I've heard of but rediscovered when looking up graph databases after reading this article. Could be a couple of solid weekends maybe with a few friends. A lot of good experience gained from my end that I could see.
- phamilton 10y agoLack of isolation and multi document atomic updates is one of the biggest difficulties. If you are updating multiple associated documents under a semantic transaction, it's possible there will be a window in which the partial update is visible. Sometimes this is harmless, other times quite harmful (changing constraints and budget on a bidding platform can result in overpaying for undesired inventory for example). This can be worked around somewhat. If you only ever update the parent documents, then you can create new associated documents and atomically update the parent document to point to all the new ones. It's a bit of a song and dance, and definitely has limitations and gets complicated fast.
- gwbas1c 10y agoI'm painfully aware of the "Lack of isolation and multi document atomic updates is one of the biggest difficulties" problem, as I encountered it when I tried working with MongoDB. (I chose MongoDB because I was working with very fluid requirements and needed a very flexible schema.) But that's not really what the article complains about! The context is that I met some of the Diaspora leads in the summer of 2010. At the time, they were ambitious, but very inexperienced. (They did teach me some valuable lessons about encryption!) The reason why I say this, is without some kind of data, it's hard to know if "don't emulate joins in MongoDB" is a conclusion of inexperience; and something that a more experienced developer would understand how to do correctly. For example, in a message board application, if a join is emulated between a discussion and user objects that just have username and avatar; the penalty of seeing an incomplete update is inconsequential. Either the user sees the old name / avatar, or the user sees the new name / avatar. So the incomplete update problem (in theory) can be handled by restricting "joins" to only when it's okay to see old data. So, assuming that the incomplete update problem is solvable, at what point do emulated joins stop scaling?
- phamilton 10y ago> by restricting "joins" to only when it's okay to see old data. So what do you do when it's not OK to see old data? That's when it breaks down. There are definitely ways around the issues, but you have to make compromises in how you build the application. If you don't make those design decisions early, you find yourself doing it wrong. In describing large scale systems, a friend of mine said "Imagine a scenario that has a one in a billion chance of happening. On a 10k qps system, it happens almost daily." That's how this breaks down at scale. Awkward corner cases that don't really seem likely suddenly become daily events.