3 ms·
More than a story about MongoDB, this is a story about hype. The unquestioning, unthinking acceptance of some technology just because someone enjoying his 15 mi
by geophile 10y ago
More than a story about MongoDB, this is a story about hype. The unquestioning, unthinking acceptance of some technology just because someone enjoying his 15 minutes of fame has tweeted about it. The designers of Diaspora set out to build a distributed database that needed to support complex queries and transactional updates. Only they didn't realize it. And they chose MongoDB because of hype. And of course it failed.
TFA says: "In 2010, when the Diaspora team was making this decision, Etsy’s articles about using document stores were quite influential, although they’ve since publicly moved away from MongoDB for data storage. Likewise, at the time, Facebook’s Cassandra was also stirring up a lot of conversation about leaving relational databases. Diaspora chose MongoDB for their social data in this zeitgeist. It was not an unreasonable choice at the time, given the information they had."
Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements, no recognition of the obvious conflict between Diaspora's data model and MongoDB's, and no discussion at all of the need for transactions, let alone distributed transactions!
TFA says: "In this post I’ve talked about how we used MongoDB vs. how it was designed to be used. I’ve talked about it as though all that information were obvious, and the Diaspora team just failed to research adequately before choosing. But this stuff wasn’t obvious at all."
Of course it was obvious. If you have a highly recursive data structure, and you decide to basically store the result of a seven-way join in a single document, then there is duplication. You should know, just from your earliest programming days, that duplicating data in data structures leads to grief. How are updates going to be handled? And if you decide against duplication, you have to do the joins, and clearly MongoDB doesn't solve that problem. So even before worrying about transactions it is obvious that you have a problem.
Finally, they migrate to MySQL and (then also? in addition?) Postgres. Although I don't quite see how this fits their database per pod architecture. I'm guessing that each pod talks to -- what -- one centralized database? Maybe they will eventually figure out what's wrong with that.
Oy.
- enraged_camel 10y ago>>Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements... You hit the nail on the head. I posit that this problem is caused by the Agile mindset that says "we don't need to collect or discuss requirements upfront. They change too often! Instead, we will just start hacking and change things as we go along." Two years later, the team is running into all kinds of complex, hard-to-reproduce problems, and they realize that they have made a terrible mistake picking MongoDB and they have to undertake a very painful migration and rewrite of the backend.
- geophile 10y agoYes, I agree, it is easy to see how Agile could lead to this particular, disastrous, dead end.