17 ms·

Why you should never use MongoDB (2013)

194 points by wheresvic1 10y ago

cromulent 10y ago
Previous discussion: https://news.ycombinator.com/item?id=6712703 https://news.ycombinator.com/item?id=6712703
SmellTheGlove 10y ago
This is from 2013 - honest question from someone that doesn't know, is it still relevant?
- hobs 10y ago
  If you read the post, the problem wasn't Mongo as much as they chose the new hotness (document db/graph model) when in reality their problem was already well solved by a RDBMS and tabular model, they had classic problems like duplicate data and performance issues when they tried to use a document db as a RDBMS. They changed over and everything was fine.
  - weddpros 10y ago
    before: mongodb is bad because it imposes we handle the case where we'd like to shard the db (which breaks joins). after: mysql is sooo much easier with joins. Who cares about sharding anyway. I hope people understand distributed systems a bit better today, but I don't have high hopes.
    - toast0 10y ago
      Without any knowledge of mongodb; if using it means you have to handle sharding right away, and using MySQL means you have to handle sharding when you outgrow a single instance, isn't that a big deal? MySQL scales pretty well these days, I think, so you can get a huge box -- a lot of stuff will fit in 2TB of ram.
      - weddpros 10y ago
        MySQL can't scale horizontally if you're using it like it's a relational database. No joins between shards means Sarah Mei could throw away her data model if she needs sharding. She would then need to denormalize her data to avoid joins, which is the main argument in the article. The project they were working on was a social network, a free alternative to Facebook. So I guess it's safe to say a single box, even the biggest you can imagine, could not ensure scalability. Beyond certain limits, sharding is the only option. Facebook has implemented exactly the denormalization despised in the article, and they're using MySQL, albeit just as a dumb key/value store ;-)
        toast0 10y ago
        I meant scale on a single box, sorry for confusion. It used to not be a great idea to get a really big SMP box to run MySQL because locking would kill performance; I think the locks are fine grained enough these days.
  - SmellTheGlove 10y ago
    OK, I read it, and that was my reaction as well, but I wasn't sure if it was due to my lack of Mongo knowledge or not. Apparently I understood it! :) I always struggle with when not to use an RDBMS, because I work with Teradata at the office and so far it hasn't let me down and there's been little pressure to find something new. And my industry (insurance) does not do new for the sake of new.
jMyles 10y ago
> Error establishing a database connection ...but this is not a reason to stop using a RDBMS, just a reason to use it in the right places and for the right things. Using it to retrieve objects to fulfill every request is not the proper way. There are caching solutions which shine at performing this task. ...just as there are applications for which a variety of key-value stores shine.
zippoxer 10y ago
I stopped reading where it said "never".
- devishard 10y ago
  I'm usually against absolutes, but I'm really in agreement here. It's almost always better to use a relational database, and in the rare cases where you wouldn't want a relational database, MongoDB is the worst of the major options. It literally memory leaks and drops data without warning or provocation. If you're not going to go with a relational database, RethinkDB, Cassandra, Redis, or BerkeleyDB would all be better choices (which one depends on situation).
  - andyana 10y ago
    Do you know which role each of those (RethinkDB, Cassandra, Redis, and BerkeleyDB) excel at?
    - willyk 10y ago
      RethinkDB is targeted at soft real time applications which have a need to update data across many clients. Additionally, although its a schemaless document DB, it offers functionality similar to joins/relational queries. This is a good presentation on RethinkDB: https://www.youtube.com/watch?v=Ee1v_SuECRk https://www.youtube.com/watch?v=Ee1v_SuECRk
    - devishard 10y ago
      I'd usually start from a problem and choose a store rather than looking at what stores excel at. So I won't speak to places where I haven't seen something solved, but here are some examples: 1. Redis is great for caching in front of a relational database, and also for running task queues. I've used it personally in both cases. Unlike MongoDB it drops data based on cache invalidation criteria, rather than dropping it randomly. Caching is one of the cases other people are proposing MongoDB for, but it's ridiculous to use MongoDB here when Redis exists. 2. Cassandra is used by Reddit and Twitter--search around and you can find lots of good writing about how they use it. Personally I've only used it indirectly via Stream.io.
weddpros 10y ago
aka. The Mongodb hater's bible EDIT: guys, be honest... if this article appears again on HN today, it's because mongodb haters are still alive and well. Downvoting my comment doesn't make the arguments in this article more valid. At least my comment downplays the importance of this article which is on HN's first page, which makes it relevant.
dangerboysteve 10y ago
I stopped reading when "Babylon 5" was used in an example.
franciscop 10y ago
Is it a joke? > Error establishing a database connection Or is it truly an error? In a page that from the title is bashing another DB
- ourcat 10y ago
  Amusingly ironic, eh? ;)
  - brazzledazzle 10y ago
    Mildly, but less so when you consider that it's a blog not a product. And even less so when you consider it likely fell over due to load and didn't silently delete the blog data.
    - cpplinuxdude 10y ago
      No matter the load. A blog is only reads. Any db system should be able to hold up. It's a different kettle of fish when there's writes; but for reads, it should be a walk through the park.
      - codyb 10y ago
        There's comments on the article, although I doubt the volume of writes there would be troublesome I have no idea really.
      - brazzledazzle 10y ago
        And yet many blog dbs fall over under what is ostensibly read-only loads. It could be inefficient connection reuse, shitty disks or it could be something as stupid as writing visitor data on each visit. But it's irrelevant to the main point which is that it's a blog. This comes up every time a blog falls over that discusses or criticizes a database on HN and it's just as shallow as it ever was. Do we really expect anyone to spend significant time optimizing their blogs for load that may not ever come? We talk about "pre-optimization" and "core competency" routinely on here and then shoot the messenger when a wordpress blog using MySQL falls over that's criticizing something like MongoDB. It's irrelevant to the point of silliness.
BukhariH 10y ago
Cached copy: https://webcache.googleusercontent.com/search?q=cache:RFqcOb8xm2EJ:www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/&num=1&hl=en&gl=uk&strip=0&vwsrc=0 https://webcache.googleusercontent.com/search?q=cache:RFqcOb...
mattlondon 10y ago
Blank page with "Error establishing database connection." Is that why you should never use MongoDB? :-)
tszming 10y ago
> Error establishing a database connection Why you should never use database
- virmundi 10y ago
  I know there is a sense of biting irony here, but it really a valid comment for relatively static sites. Look at the Git based systems out there. If you can generate your site without a tier, generate it without a tier.
alex95 10y ago
Your "I'm better at it than you are" is childish. I can't take you seriously after you post something like this. Furthermore, again, as stated in the previous discussion, using something badly and not learning the ropes, but then claiming such an awful title is joke-worthy. "Why you should never use MongoDB" - You mean "Why you should never use MongoDB if you aren't capable of learning how to use it correctly". Infact, just because it doesn't fit your instance doesn't mean it doesn't fit another persons instance. Awful article.
avitzurel 10y ago
I think it's ironic that the page says "Error establishing a database connection". As someone who used MongoDB extensively in the past (8TB+ of data) and also managed the devops side of things, I can tell you straight out that MongoDB has a place in a lot of startup stacks. I would likely not use it as a main source of truth for any application. However for a lot of things, it's a good database. Since I can't read the post I can't really address the points made in it, so at this point enough said.
- devishard 10y ago
  > I would likely not use it as a main source of truth for any application but for a lot of things, it's a good database. I really don't understand this. In what case is it every acceptable for a data store to lose data? And that's not even MongoDB's only problem: it memory leaks!
  - avitzurel 10y ago
    I have to say that I did not experience a single data loss that was a result of the database misbehaving. Memory leaks weren't a huge issue for us as well. After stabilizing the setup I have to say it was basically a fire and forget part of the stack for us. The role of MongoDB was to act as a fast-insert and aggregation framework for other parts of the system. So, we would insert BIG amounts of data at a time and aggregate it into a K/V store where we pulled the data from. After a while, the setup became to expensive to run, at this point we turned it off for a cheaper solution, but in terms of functionality, it functioned pretty well.
    - mack73 10y ago
      Fire-and-forget is awesome for when you do not care at all about your data. I'm sure those types of writes are super (duper) fast. What exactly is the use case for that? Serious question.
      - vidarh 10y ago
        Don't know about the guy you replied to, but e.g. consider any application that regularly crawl feeds, api's etc. where the data is rapidly changing and only a portion of the data is necessary to give good output. There are lots of applications like that where you just need "enough" data to give good results and/or where any loss will auto-heal next time you crawl the original source.
Ridikule 10y ago
"Error establishing a database connection" The irony is thick.
ThePhysicist 10y ago
The post should probably be titled "Why you should not pick a technology based on hype and without evaluating it first". While MongoDB has (and probably still has) some flaws and is not the perfect DB system, there are valid use cases where it can be a good choice. Building a social networking site that requires rich queries along a relationship graph is most definitely not one of them.
- enraged_camel 10y ago
  The HN consensus seems to be that MongoDB is good only for toy projects, and that you should switch to a real database as soon as things start getting more complex.
  - rch 10y ago
    Actually, when you have a toy project that needs a RDBMS, go ahead and use SQLite. The consensus is that relatively few applications are a good fit for MongoDB, irrespective of scale.
    - tensor 10y ago
      And for those few where you think MongoDB might be a good fit, you should probably use something like RethinkDB instead.
    - honkhonkpants 10y ago
      Or leveldb for those even more trivial storage needs.
      - deleted 10y ago
        [deleted]
    - 3pt14159 10y ago
      Why? Now I have to learn a crappier version of SQL (no support for real date types, no user management, etc) and heaven help me if my toy project actual gets traction. I'll have to migrate everything over to Postgres or I'll have to keep my DB on a single server because it's just a single file. Compared to just using Postgres or even MySQL I do not understand the benefit. It takes less than an hour to spin up a PG database with user permissions and migrations to make tables and indexes.
NathanKP 10y ago
Ironic: http://i.imgur.com/RGApTVz.png http://i.imgur.com/RGApTVz.png
pmelendez 10y ago
Tldr; They tried to use a relational data model in a document database and failed. I am not sure the author realize that a relational model is actually a graph.
chatmasta 10y ago
My first introduction to databases was with PHP/MySQL, where normalization was the name of the game. The whole point of normalization is that there is no duplication of data anywhere. If it's possible for duplicate data to exist, that's a symptom of a design flaw in the schema. I've been using Mongo recently, and every time I raise criticism of it, the counterargument I hear is "forget about normalization! Duplication is okay." Like the author, I really cannot wrap my head around this. While I understand that duplicating documents across collections may make querying faster, what about when you want to change the document? You need to propagate the change across every duplicate of the document in every collection where it exists. This means that any "de-duplication" logic needs to happen at the application level, rather than the database level. Parse.com provided an interesting abstraction around that with "cloud code" and beforeSave and afterSave triggers. To maintain consistency, they encourage propagating updates to a document within its collection's beforeSave function. So if a document changes in one collection, you write code to change all instances of the document in other collections. That's nice in that it almost feels like you're writing the dedeuplication logic in the database layer, because you can view the "beforeSave" and "afterSave" functions as extensions of the schema. As long as those functions are up to date, the schema and any pseudo-linked documents will stay up to date. But I really don't buy it. The strengths of mongo encourage a design that necessitates complexity for any significant write operation. I think the real issue is that "if you have a hammer, everything looks like a nail." Mongo and other NoSQL stores have some real use cases, but people who are more familiar with Mongo than RDBMS are too trigger happy to employ it as a solution to problems where a RDBMS is the clear solution.
- fiatjaf 10y ago
  It has always seemed to me that the solution to the duplication/normalization dilemma is an approach that gets "normalized" data (not necessarily normalized in the sense that it is scattered among many tables, but in the sense that there is a single source of truth for each data point) and uses it to write materialized views, automatically and efficiently recomputed and stored for fast querying. CouchDB does this to a certain extent, but its JSON-document approach is not appropriate enough.
- overcast 10y ago
andrewclunn 10y ago
An interesting article. I'm always wary with no approaches in technology due to the "trendy" factor that often pushes for the novel for no reason other than its novelty. That said, I am curious if there are use cases where non-relational databases like MongoDB make sense as data caches for quick read only access, since the data could be formatted in the way that API calls would already expect it to be post extraction.
gwbas1c 10y ago
It takes that article a long time to get to the point. What I'd like to know is, "why is emulating joins bad?" Specifically, why is it bad to load a MongoDB document, and then load downstream documents that it links to? What kind of problems does this lead to? Granted, when I worked with MongoDB, I encountered problems due to its lack of transactions. (It's surprisingly unreliable if you end up needing to update multiple documents; when a relational database can do that easily in a transaction.) But, assuming you can pick an application design that does single-document updates; why is following links in a document bad?
- codyb 10y ago
  Can you do it at the database level? Since MDB doesn't have joins wouldn't emulation require a lot more transactions across the wire? And in a distributed system couldn't that add up very quickly?
  - gwbas1c 10y ago
    I'm looking for a more quantitative answer. For example, if I write a message board in MDB: - An entire discussion (think hacker news story and all corresponding discussion) is a single document - Each post, reply, ect, has a user ID - I load the post object and then do a "where uid in (all uids in document)" query Obviously, that has scalability limits, but what are they? In practice, when do you hit them?
    - codyb 10y ago
      Sounds like a great article. Take some of the open sourced reddit comments, or just create a small script to generate lorem ipsum posts for "users". Deploy mongo somewhere. Scale the data and scale the number of requests per second and see when things fall off. I've got so many projects I'd like to do but that does sound neat. I'd like to compare it to couchbase and postgres as well, two databases I'm much more interested in, and also Neo4j which I'm sure I've heard of but rediscovered when looking up graph databases after reading this article. Could be a couple of solid weekends maybe with a few friends. A lot of good experience gained from my end that I could see.
mack73 10y ago
At this point in time, making a statement about MongoDB on HN should almost be considered trolling. We already know about the short-comings of that code-base. It will work fine until it doesn't and you'll loose some of your data. Some folks are using it and enjoying it. Others aren't. "Your shouldn't use it" is like saying you shouldn't use javascript for things other than UI. Noone cares anymore.
- brandur 10y ago
  To be fair, I think your comment shows why this sort of post _can_ still be useful. You're very unlikely to lose data on a modern Mongo system using default configuration (their troubles with durability have largely been solved), but there are many other good reasons not to use it, and it's enlightening to read about and understand what they are.
  - mack73 10y ago
    If "their troubles with durability have largely been solved" were true then I'm sure HN would be flooded with posts about "MongoDB solves durability with an entirely new architecture". I might be wrong here. How did they solve the "your data is lost when a disk fails"?
    - brandur 10y ago
      I'm talking about "durability" here in the context of the "D" in "ACID". Previously, Mongo had very serious problems in that are because its client would assume that any message sent to the outgoing socket buffer was persisted "well enough", which was an obvious untruth [1]. This was also one of the somewhat underhanded techniques they used to achieve their early benchmarks. As of version 3, they have defaulted their client's "write_concern" value to "1", which means that it will wait for confirmation from a replica set's primary before considering a value persisted [2]. This puts Mongo roughly on the same level as any other database in terms of durability guarantees. Disk failure is entirely tangential to your original premise that "Mongo loses data". I'm not getting into it, but there a variety of techniques that Mongo (and every other known database) can use to protect against that. [1] http://hackingdistributed.com/2013/01/29/mongo-ft/ http://hackingdistributed.com/2013/01/29/mongo-ft/ [2] https://docs.mongodb.com/manual/reference/write-concern/ https://docs.mongodb.com/manual/reference/write-concern/
cabalamat 10y ago
I got "Error establishing a database connection", maybe this wouldn't have happened if they had used MongoDB :-)
- danpalmer 10y ago
  Yeah, the post would have never even have been saved to disk in the first place :P
geophile 10y ago
More than a story about MongoDB, this is a story about hype. The unquestioning, unthinking acceptance of some technology just because someone enjoying his 15 minutes of fame has tweeted about it. The designers of Diaspora set out to build a distributed database that needed to support complex queries and transactional updates. Only they didn't realize it. And they chose MongoDB because of hype. And of course it failed. TFA says: "In 2010, when the Diaspora team was making this decision, Etsy’s articles about using document stores were quite influential, although they’ve since publicly moved away from MongoDB for data storage. Likewise, at the time, Facebook’s Cassandra was also stirring up a lot of conversation about leaving relational databases. Diaspora chose MongoDB for their social data in this zeitgeist. It was not an unreasonable choice at the time, given the information they had." Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements, no recognition of the obvious conflict between Diaspora's data model and MongoDB's, and no discussion at all of the need for transactions, let alone distributed transactions! TFA says: "In this post I’ve talked about how we used MongoDB vs. how it was designed to be used. I’ve talked about it as though all that information were obvious, and the Diaspora team just failed to research adequately before choosing. But this stuff wasn’t obvious at all." Of course it was obvious. If you have a highly recursive data structure, and you decide to basically store the result of a seven-way join in a single document, then there is duplication. You should know, just from your earliest programming days, that duplicating data in data structures leads to grief. How are updates going to be handled? And if you decide against duplication, you have to do the joins, and clearly MongoDB doesn't solve that problem. So even before worrying about transactions it is obvious that you have a problem. Finally, they migrate to MySQL and (then also? in addition?) Postgres. Although I don't quite see how this fits their database per pod architecture. I'm guessing that each pod talks to -- what -- one centralized database? Maybe they will eventually figure out what's wrong with that. Oy.
- enraged_camel 10y ago
  >>Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements... You hit the nail on the head. I posit that this problem is caused by the Agile mindset that says "we don't need to collect or discuss requirements upfront. They change too often! Instead, we will just start hacking and change things as we go along." Two years later, the team is running into all kinds of complex, hard-to-reproduce problems, and they realize that they have made a terrible mistake picking MongoDB and they have to undertake a very painful migration and rewrite of the backend.
electricEmu 10y ago
This article has little to do with the specific failings of MongoDB at all. The author takes offense with document databases and denormalization. I disagree with the blanket statement. Denormalization. It's not always the answer, but sometimes making reads easy and writes more difficult isn't bad for the problem set. The funny thing is a social network at scale isbt going to be able to use a traditional SQL database without sharding it and killing joins. It will be a documebt DB with limited SQL features. As for Mongo, it is sold as the answer to everything just like any other product. It actually fits some problems sets. I actually had great success with it, at scale, in an Amazon division. It ain't the answer for a lot of problem sets though. I give this blog post two of five for bringing up a mildly interesting topic HN already beat to the ground with misguided conclusions.
nailer 10y ago
I like document oriented databases. It seems odd I should need to change the data structure of an item in order to persist it. After about 5 years of the many/varied cases of Mongo losing data, I switched to rethinkdb for my current project. It's an uninteresting, surprise-less database and I like it very much.
manishsharan 10y ago
Not this FUD again! It is true that MongoDB created a lot of unexpected problems for the early adopters who did not do enough due diligence and testing. I encountered them during my POC with Mongodb but I was able to get past them easily thanks to a robust user community. I have experiences no such issues with current version with Mongodb. Today I am fighting my enterprise bureaucracy to get MOngoDB for our enterprise applications .FUD like this allows armchair architects to quote this article to spook the management and we are stuck with stuffing json as in a RDBMS tables.
- fadzlan 10y ago
  The article is from 2013, so its very much likely that some of the issues has been handled by MongoDB itself or some agreed practices.
- hitchhiker999 10y ago
  Finally, great comment - had to scroll too far to get to this! 1) The developers clearly weren't experienced enough to know what document storage is and when/where it shines/fails. 2) Author jumped on the ever-boring 'i hate mongo' easy train. 3) I am so sick of technology becoming a religious war.
jmccay 10y ago
This article was posted almost three years ago and went through the same love/hate cycle on Hacker News then. Why the repeat? I am sure a number of technical points could be outdated by now.
spynxic 10y ago
What about TokuMX? -- https://www.percona.com/software/mongo-database/percona-tokumx https://www.percona.com/software/mongo-database/percona-toku...
hitr 10y ago
I ran into exactly the same problem as the author . I started working on mongo as it was already chosen as backend for a startup .i tried to avoid it but they did not listen. We understood duplication is the only way forward and you need to unlearn your RDBS skill.Then features crop in and you realize that you need joins badly.Then you would like to have some kind of transaction support and its high time you realize you are really screwed. I always used to wonder what is the real use case for mongo and that is explained nicely in the blog The only thing it’s good at is storing arbitrary pieces of JSON. “Arbitrary,” in this context, means that you don’t care at all what’s inside that JSON. You don’t even look. There is no schema, not even an implicit schema
coding123 10y ago
ArangoDB would have been a good fit, it's basically mongo db with graph features. By the way 99.99999% of the time a WordPress blog can't establish a database connection is because they used mysql.
- feld 10y ago
  The database backend of a Wordpress blog has nothing to do with this conversation. Any software can be poorly tuned/configured. NoSQL wouldn't magically solve this problem.
ChicagoDave 10y ago
I would have switched to a graph database like neo4j. Most if not all of your problems would be solved.
deleted 10y ago
[deleted]
codingdave 10y ago
I've always found that any statement that generalizes why you should never use a tool is actually just someone who has not grokked the tool enough to understand its appropriate use cases.
simonebrunozzi 10y ago
I don't know if the author is reading this, but the "fault tolerance" picture of the two coffee machines is something that I have used when I was at AWS. Curious to know if he was somewhat inspired by it.
20years 10y ago
My company got hired for 2 very large MongoDB to MySQL migration projects this year alone totaling over $170k. These apps should have never been on MongoDB in the first place. Total mess that cost them lots of $$. I am okay if others continue to use MongoDB. It will keep my team gainfully employed ;)
- icc97 10y ago
  Wow, that is truly impressive. I'd be surprised if there's any examples of people prepared to spend the same amount of money to go the opposite way.
  - 20years 10y ago
    To be fair, the projects pretty much required a full code re-write too. MongoDB in both cases was one of many bad decisions. Choosing all hipster stuff is what got them into trouble.
- pbreit 10y ago
  If you're gonna migrate to SQL, wouldn't Postgres be a better call?
  - 20years 10y ago
    Postgres is great and was on the table but MySQL was chosen for a variety of reasons. In particular, one of the companies already had a lot of other things running on MySQL that they wanted integrated. They were happy with that setup and MySQL was a perfectly fine choice for the app we were migrating from MongoDB.
yongjik 10y ago
> Here we have copies of user data inlined. This is Joe’s stream, and it has a copy of his user data, including his name and URL, at the top level. His stream, just underneath, contains Jane’s post. Joe has liked Jane’s post, so under likes for Jane’s post, we have a separate copy of Joe’s data. > You can see why this is attractive: all the data you need is already located where you need it. That doesn't sound attractive at all. That sounds more like a recipe for "we use five hundred times more RAM than the deduplicated data that we actually have" type of disaster. ...All for a production data set which could be "turned into about 1.2 million rows in MySQL." I think the moral is "If you think your data is too big for a relational database, you're almost certainly wrong."
GickRimes 10y ago
"Shards are the secret ingredient in the webscale sauce, they just work.." :) https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs
nnain 10y ago
However, MongoDB seems to be marching on fine in comparison to other popular NoSQL DBs (viz. CouchDB and RethinkDB) for the past decade - https://www.google.com/trends/explore?date=2006-08-16%202016-08-16&q=MongoDB,CouchDB,RethinkDB https://www.google.com/trends/explore?date=2006-08-16%202016...
danjc 10y ago
Non-relational database for a million rows of data? Madness.

194 points by wheresvic1 10y ago

cromulent 10y ago

Previous discussion: https://news.ycombinator.com/item?id=6712703 https://news.ycombinator.com/item?id=6712703

SmellTheGlove 10y ago

This is from 2013 - honest question from someone that doesn't know, is it still relevant?

hobs 10y ago
If you read the post, the problem wasn't Mongo as much as they chose the new hotness (document db/graph model) when in reality their problem was already well solved by a RDBMS and tabular model, they had classic problems like duplicate data and performance issues when they tried to use a document db as a RDBMS. They changed over and everything was fine.
- weddpros 10y ago
  before: mongodb is bad because it imposes we handle the case where we'd like to shard the db (which breaks joins). after: mysql is sooo much easier with joins. Who cares about sharding anyway. I hope people understand distributed systems a bit better today, but I don't have high hopes.
  - toast0 10y ago
    Without any knowledge of mongodb; if using it means you have to handle sharding right away, and using MySQL means you have to handle sharding when you outgrow a single instance, isn't that a big deal? MySQL scales pretty well these days, I think, so you can get a huge box -- a lot of stuff will fit in 2TB of ram.
    - weddpros 10y ago
      MySQL can't scale horizontally if you're using it like it's a relational database. No joins between shards means Sarah Mei could throw away her data model if she needs sharding. She would then need to denormalize her data to avoid joins, which is the main argument in the article. The project they were working on was a social network, a free alternative to Facebook. So I guess it's safe to say a single box, even the biggest you can imagine, could not ensure scalability. Beyond certain limits, sharding is the only option. Facebook has implemented exactly the denormalization despised in the article, and they're using MySQL, albeit just as a dumb key/value store ;-)
      - toast0 10y ago
        I meant scale on a single box, sorry for confusion. It used to not be a great idea to get a really big SMP box to run MySQL because locking would kill performance; I think the locks are fine grained enough these days.
- SmellTheGlove 10y ago
  OK, I read it, and that was my reaction as well, but I wasn't sure if it was due to my lack of Mongo knowledge or not. Apparently I understood it! :) I always struggle with when not to use an RDBMS, because I work with Teradata at the office and so far it hasn't let me down and there's been little pressure to find something new. And my industry (insurance) does not do new for the sake of new.

jMyles 10y ago

> Error establishing a database connection ...but this is not a reason to stop using a RDBMS, just a reason to use it in the right places and for the right things. Using it to retrieve objects to fulfill every request is not the proper way. There are caching solutions which shine at performing this task. ...just as there are applications for which a variety of key-value stores shine.

zippoxer 10y ago

I stopped reading where it said "never".

devishard 10y ago
I'm usually against absolutes, but I'm really in agreement here. It's almost always better to use a relational database, and in the rare cases where you wouldn't want a relational database, MongoDB is the worst of the major options. It literally memory leaks and drops data without warning or provocation. If you're not going to go with a relational database, RethinkDB, Cassandra, Redis, or BerkeleyDB would all be better choices (which one depends on situation).
- andyana 10y ago
  Do you know which role each of those (RethinkDB, Cassandra, Redis, and BerkeleyDB) excel at?
  - willyk 10y ago
    RethinkDB is targeted at soft real time applications which have a need to update data across many clients. Additionally, although its a schemaless document DB, it offers functionality similar to joins/relational queries. This is a good presentation on RethinkDB: https://www.youtube.com/watch?v=Ee1v_SuECRk https://www.youtube.com/watch?v=Ee1v_SuECRk
  - devishard 10y ago
    I'd usually start from a problem and choose a store rather than looking at what stores excel at. So I won't speak to places where I haven't seen something solved, but here are some examples: 1. Redis is great for caching in front of a relational database, and also for running task queues. I've used it personally in both cases. Unlike MongoDB it drops data based on cache invalidation criteria, rather than dropping it randomly. Caching is one of the cases other people are proposing MongoDB for, but it's ridiculous to use MongoDB here when Redis exists. 2. Cassandra is used by Reddit and Twitter--search around and you can find lots of good writing about how they use it. Personally I've only used it indirectly via Stream.io.

weddpros 10y ago

aka. The Mongodb hater's bible EDIT: guys, be honest... if this article appears again on HN today, it's because mongodb haters are still alive and well. Downvoting my comment doesn't make the arguments in this article more valid. At least my comment downplays the importance of this article which is on HN's first page, which makes it relevant.

dangerboysteve 10y ago

I stopped reading when "Babylon 5" was used in an example.

franciscop 10y ago

Is it a joke? > Error establishing a database connection Or is it truly an error? In a page that from the title is bashing another DB

ourcat 10y ago
Amusingly ironic, eh? ;)
- brazzledazzle 10y ago
  Mildly, but less so when you consider that it's a blog not a product. And even less so when you consider it likely fell over due to load and didn't silently delete the blog data.
  - cpplinuxdude 10y ago
    No matter the load. A blog is only reads. Any db system should be able to hold up. It's a different kettle of fish when there's writes; but for reads, it should be a walk through the park.
    - codyb 10y ago
      There's comments on the article, although I doubt the volume of writes there would be troublesome I have no idea really.
    - brazzledazzle 10y ago
      And yet many blog dbs fall over under what is ostensibly read-only loads. It could be inefficient connection reuse, shitty disks or it could be something as stupid as writing visitor data on each visit. But it's irrelevant to the main point which is that it's a blog. This comes up every time a blog falls over that discusses or criticizes a database on HN and it's just as shallow as it ever was. Do we really expect anyone to spend significant time optimizing their blogs for load that may not ever come? We talk about "pre-optimization" and "core competency" routinely on here and then shoot the messenger when a wordpress blog using MySQL falls over that's criticizing something like MongoDB. It's irrelevant to the point of silliness.

BukhariH 10y ago

Cached copy: https://webcache.googleusercontent.com/search?q=cache:RFqcOb8xm2EJ:www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/&num=1&hl=en&gl=uk&strip=0&vwsrc=0 https://webcache.googleusercontent.com/search?q=cache:RFqcOb...

mattlondon 10y ago

Blank page with "Error establishing database connection." Is that why you should never use MongoDB? :-)

tszming 10y ago

> Error establishing a database connection Why you should never use database

virmundi 10y ago
I know there is a sense of biting irony here, but it really a valid comment for relatively static sites. Look at the Git based systems out there. If you can generate your site without a tier, generate it without a tier.

alex95 10y ago

Your "I'm better at it than you are" is childish. I can't take you seriously after you post something like this. Furthermore, again, as stated in the previous discussion, using something badly and not learning the ropes, but then claiming such an awful title is joke-worthy. "Why you should never use MongoDB" - You mean "Why you should never use MongoDB if you aren't capable of learning how to use it correctly". Infact, just because it doesn't fit your instance doesn't mean it doesn't fit another persons instance. Awful article.

avitzurel 10y ago

I think it's ironic that the page says "Error establishing a database connection". As someone who used MongoDB extensively in the past (8TB+ of data) and also managed the devops side of things, I can tell you straight out that MongoDB has a place in a lot of startup stacks. I would likely not use it as a main source of truth for any application. However for a lot of things, it's a good database. Since I can't read the post I can't really address the points made in it, so at this point enough said.

devishard 10y ago
> I would likely not use it as a main source of truth for any application but for a lot of things, it's a good database. I really don't understand this. In what case is it every acceptable for a data store to lose data? And that's not even MongoDB's only problem: it memory leaks!
- avitzurel 10y ago
  I have to say that I did not experience a single data loss that was a result of the database misbehaving. Memory leaks weren't a huge issue for us as well. After stabilizing the setup I have to say it was basically a fire and forget part of the stack for us. The role of MongoDB was to act as a fast-insert and aggregation framework for other parts of the system. So, we would insert BIG amounts of data at a time and aggregate it into a K/V store where we pulled the data from. After a while, the setup became to expensive to run, at this point we turned it off for a cheaper solution, but in terms of functionality, it functioned pretty well.
  - mack73 10y ago
    Fire-and-forget is awesome for when you do not care at all about your data. I'm sure those types of writes are super (duper) fast. What exactly is the use case for that? Serious question.
    - vidarh 10y ago
      Don't know about the guy you replied to, but e.g. consider any application that regularly crawl feeds, api's etc. where the data is rapidly changing and only a portion of the data is necessary to give good output. There are lots of applications like that where you just need "enough" data to give good results and/or where any loss will auto-heal next time you crawl the original source.

Ridikule 10y ago

"Error establishing a database connection" The irony is thick.

ThePhysicist 10y ago

The post should probably be titled "Why you should not pick a technology based on hype and without evaluating it first". While MongoDB has (and probably still has) some flaws and is not the perfect DB system, there are valid use cases where it can be a good choice. Building a social networking site that requires rich queries along a relationship graph is most definitely not one of them.

enraged_camel 10y ago
The HN consensus seems to be that MongoDB is good only for toy projects, and that you should switch to a real database as soon as things start getting more complex.
- rch 10y ago
  Actually, when you have a toy project that needs a RDBMS, go ahead and use SQLite. The consensus is that relatively few applications are a good fit for MongoDB, irrespective of scale.
  - tensor 10y ago
    And for those few where you think MongoDB might be a good fit, you should probably use something like RethinkDB instead.
  - honkhonkpants 10y ago
    Or leveldb for those even more trivial storage needs.
    - deleted 10y ago
      [deleted]
  - 3pt14159 10y ago
    Why? Now I have to learn a crappier version of SQL (no support for real date types, no user management, etc) and heaven help me if my toy project actual gets traction. I'll have to migrate everything over to Postgres or I'll have to keep my DB on a single server because it's just a single file. Compared to just using Postgres or even MySQL I do not understand the benefit. It takes less than an hour to spin up a PG database with user permissions and migrations to make tables and indexes.

NathanKP 10y ago

Ironic: http://i.imgur.com/RGApTVz.png http://i.imgur.com/RGApTVz.png

pmelendez 10y ago

Tldr; They tried to use a relational data model in a document database and failed. I am not sure the author realize that a relational model is actually a graph.

chatmasta 10y ago

My first introduction to databases was with PHP/MySQL, where normalization was the name of the game. The whole point of normalization is that there is no duplication of data anywhere. If it's possible for duplicate data to exist, that's a symptom of a design flaw in the schema. I've been using Mongo recently, and every time I raise criticism of it, the counterargument I hear is "forget about normalization! Duplication is okay." Like the author, I really cannot wrap my head around this. While I understand that duplicating documents across collections may make querying faster, what about when you want to change the document? You need to propagate the change across every duplicate of the document in every collection where it exists. This means that any "de-duplication" logic needs to happen at the application level, rather than the database level. Parse.com provided an interesting abstraction around that with "cloud code" and beforeSave and afterSave triggers. To maintain consistency, they encourage propagating updates to a document within its collection's beforeSave function. So if a document changes in one collection, you write code to change all instances of the document in other collections. That's nice in that it almost feels like you're writing the dedeuplication logic in the database layer, because you can view the "beforeSave" and "afterSave" functions as extensions of the schema. As long as those functions are up to date, the schema and any pseudo-linked documents will stay up to date. But I really don't buy it. The strengths of mongo encourage a design that necessitates complexity for any significant write operation. I think the real issue is that "if you have a hammer, everything looks like a nail." Mongo and other NoSQL stores have some real use cases, but people who are more familiar with Mongo than RDBMS are too trigger happy to employ it as a solution to problems where a RDBMS is the clear solution.

fiatjaf 10y ago
It has always seemed to me that the solution to the duplication/normalization dilemma is an approach that gets "normalized" data (not necessarily normalized in the sense that it is scattered among many tables, but in the sense that there is a single source of truth for each data point) and uses it to write materialized views, automatically and efficiently recomputed and stored for fast querying. CouchDB does this to a certain extent, but its JSON-document approach is not appropriate enough.
overcast 10y ago

andrewclunn 10y ago

An interesting article. I'm always wary with no approaches in technology due to the "trendy" factor that often pushes for the novel for no reason other than its novelty. That said, I am curious if there are use cases where non-relational databases like MongoDB make sense as data caches for quick read only access, since the data could be formatted in the way that API calls would already expect it to be post extraction.

gwbas1c 10y ago

It takes that article a long time to get to the point. What I'd like to know is, "why is emulating joins bad?" Specifically, why is it bad to load a MongoDB document, and then load downstream documents that it links to? What kind of problems does this lead to? Granted, when I worked with MongoDB, I encountered problems due to its lack of transactions. (It's surprisingly unreliable if you end up needing to update multiple documents; when a relational database can do that easily in a transaction.) But, assuming you can pick an application design that does single-document updates; why is following links in a document bad?

codyb 10y ago
Can you do it at the database level? Since MDB doesn't have joins wouldn't emulation require a lot more transactions across the wire? And in a distributed system couldn't that add up very quickly?
- gwbas1c 10y ago
  I'm looking for a more quantitative answer. For example, if I write a message board in MDB: - An entire discussion (think hacker news story and all corresponding discussion) is a single document - Each post, reply, ect, has a user ID - I load the post object and then do a "where uid in (all uids in document)" query Obviously, that has scalability limits, but what are they? In practice, when do you hit them?
  - codyb 10y ago
    Sounds like a great article. Take some of the open sourced reddit comments, or just create a small script to generate lorem ipsum posts for "users". Deploy mongo somewhere. Scale the data and scale the number of requests per second and see when things fall off. I've got so many projects I'd like to do but that does sound neat. I'd like to compare it to couchbase and postgres as well, two databases I'm much more interested in, and also Neo4j which I'm sure I've heard of but rediscovered when looking up graph databases after reading this article. Could be a couple of solid weekends maybe with a few friends. A lot of good experience gained from my end that I could see.

mack73 10y ago

At this point in time, making a statement about MongoDB on HN should almost be considered trolling. We already know about the short-comings of that code-base. It will work fine until it doesn't and you'll loose some of your data. Some folks are using it and enjoying it. Others aren't. "Your shouldn't use it" is like saying you shouldn't use javascript for things other than UI. Noone cares anymore.

brandur 10y ago
To be fair, I think your comment shows why this sort of post _can_ still be useful. You're very unlikely to lose data on a modern Mongo system using default configuration (their troubles with durability have largely been solved), but there are many other good reasons not to use it, and it's enlightening to read about and understand what they are.
- mack73 10y ago
  If "their troubles with durability have largely been solved" were true then I'm sure HN would be flooded with posts about "MongoDB solves durability with an entirely new architecture". I might be wrong here. How did they solve the "your data is lost when a disk fails"?
  - brandur 10y ago
    I'm talking about "durability" here in the context of the "D" in "ACID". Previously, Mongo had very serious problems in that are because its client would assume that any message sent to the outgoing socket buffer was persisted "well enough", which was an obvious untruth [1]. This was also one of the somewhat underhanded techniques they used to achieve their early benchmarks. As of version 3, they have defaulted their client's "write_concern" value to "1", which means that it will wait for confirmation from a replica set's primary before considering a value persisted [2]. This puts Mongo roughly on the same level as any other database in terms of durability guarantees. Disk failure is entirely tangential to your original premise that "Mongo loses data". I'm not getting into it, but there a variety of techniques that Mongo (and every other known database) can use to protect against that. [1] http://hackingdistributed.com/2013/01/29/mongo-ft/ http://hackingdistributed.com/2013/01/29/mongo-ft/ [2] https://docs.mongodb.com/manual/reference/write-concern/ https://docs.mongodb.com/manual/reference/write-concern/

cabalamat 10y ago

I got "Error establishing a database connection", maybe this wouldn't have happened if they had used MongoDB :-)

danpalmer 10y ago
Yeah, the post would have never even have been saved to disk in the first place :P

geophile 10y ago

More than a story about MongoDB, this is a story about hype. The unquestioning, unthinking acceptance of some technology just because someone enjoying his 15 minutes of fame has tweeted about it. The designers of Diaspora set out to build a distributed database that needed to support complex queries and transactional updates. Only they didn't realize it. And they chose MongoDB because of hype. And of course it failed. TFA says: "In 2010, when the Diaspora team was making this decision, Etsy’s articles about using document stores were quite influential, although they’ve since publicly moved away from MongoDB for data storage. Likewise, at the time, Facebook’s Cassandra was also stirring up a lot of conversation about leaving relational databases. Diaspora chose MongoDB for their social data in this zeitgeist. It was not an unreasonable choice at the time, given the information they had." Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements, no recognition of the obvious conflict between Diaspora's data model and MongoDB's, and no discussion at all of the need for transactions, let alone distributed transactions! TFA says: "In this post I’ve talked about how we used MongoDB vs. how it was designed to be used. I’ve talked about it as though all that information were obvious, and the Diaspora team just failed to research adequately before choosing. But this stuff wasn’t obvious at all." Of course it was obvious. If you have a highly recursive data structure, and you decide to basically store the result of a seven-way join in a single document, then there is duplication. You should know, just from your earliest programming days, that duplicating data in data structures leads to grief. How are updates going to be handled? And if you decide against duplication, you have to do the joins, and clearly MongoDB doesn't solve that problem. So even before worrying about transactions it is obvious that you have a problem. Finally, they migrate to MySQL and (then also? in addition?) Postgres. Although I don't quite see how this fits their database per pod architecture. I'm guessing that each pod talks to -- what -- one centralized database? Maybe they will eventually figure out what's wrong with that. Oy.

enraged_camel 10y ago
>>Yes, it was an unreasonable choice. This is basically saying that the cool kids are using document stores, so we should too. There is no discussion of requirements... You hit the nail on the head. I posit that this problem is caused by the Agile mindset that says "we don't need to collect or discuss requirements upfront. They change too often! Instead, we will just start hacking and change things as we go along." Two years later, the team is running into all kinds of complex, hard-to-reproduce problems, and they realize that they have made a terrible mistake picking MongoDB and they have to undertake a very painful migration and rewrite of the backend.

electricEmu 10y ago

This article has little to do with the specific failings of MongoDB at all. The author takes offense with document databases and denormalization. I disagree with the blanket statement. Denormalization. It's not always the answer, but sometimes making reads easy and writes more difficult isn't bad for the problem set. The funny thing is a social network at scale isbt going to be able to use a traditional SQL database without sharding it and killing joins. It will be a documebt DB with limited SQL features. As for Mongo, it is sold as the answer to everything just like any other product. It actually fits some problems sets. I actually had great success with it, at scale, in an Amazon division. It ain't the answer for a lot of problem sets though. I give this blog post two of five for bringing up a mildly interesting topic HN already beat to the ground with misguided conclusions.

nailer 10y ago

I like document oriented databases. It seems odd I should need to change the data structure of an item in order to persist it. After about 5 years of the many/varied cases of Mongo losing data, I switched to rethinkdb for my current project. It's an uninteresting, surprise-less database and I like it very much.

manishsharan 10y ago

Not this FUD again! It is true that MongoDB created a lot of unexpected problems for the early adopters who did not do enough due diligence and testing. I encountered them during my POC with Mongodb but I was able to get past them easily thanks to a robust user community. I have experiences no such issues with current version with Mongodb. Today I am fighting my enterprise bureaucracy to get MOngoDB for our enterprise applications .FUD like this allows armchair architects to quote this article to spook the management and we are stuck with stuffing json as in a RDBMS tables.

fadzlan 10y ago
The article is from 2013, so its very much likely that some of the issues has been handled by MongoDB itself or some agreed practices.
hitchhiker999 10y ago
Finally, great comment - had to scroll too far to get to this! 1) The developers clearly weren't experienced enough to know what document storage is and when/where it shines/fails. 2) Author jumped on the ever-boring 'i hate mongo' easy train. 3) I am so sick of technology becoming a religious war.

jmccay 10y ago

This article was posted almost three years ago and went through the same love/hate cycle on Hacker News then. Why the repeat? I am sure a number of technical points could be outdated by now.

spynxic 10y ago

What about TokuMX? -- https://www.percona.com/software/mongo-database/percona-tokumx https://www.percona.com/software/mongo-database/percona-toku...

hitr 10y ago

I ran into exactly the same problem as the author . I started working on mongo as it was already chosen as backend for a startup .i tried to avoid it but they did not listen. We understood duplication is the only way forward and you need to unlearn your RDBS skill.Then features crop in and you realize that you need joins badly.Then you would like to have some kind of transaction support and its high time you realize you are really screwed. I always used to wonder what is the real use case for mongo and that is explained nicely in the blog The only thing it’s good at is storing arbitrary pieces of JSON. “Arbitrary,” in this context, means that you don’t care at all what’s inside that JSON. You don’t even look. There is no schema, not even an implicit schema

coding123 10y ago

ArangoDB would have been a good fit, it's basically mongo db with graph features. By the way 99.99999% of the time a WordPress blog can't establish a database connection is because they used mysql.

feld 10y ago
The database backend of a Wordpress blog has nothing to do with this conversation. Any software can be poorly tuned/configured. NoSQL wouldn't magically solve this problem.

ChicagoDave 10y ago

I would have switched to a graph database like neo4j. Most if not all of your problems would be solved.

deleted 10y ago

[deleted]

codingdave 10y ago

I've always found that any statement that generalizes why you should never use a tool is actually just someone who has not grokked the tool enough to understand its appropriate use cases.

simonebrunozzi 10y ago

I don't know if the author is reading this, but the "fault tolerance" picture of the two coffee machines is something that I have used when I was at AWS. Curious to know if he was somewhat inspired by it.

20years 10y ago

My company got hired for 2 very large MongoDB to MySQL migration projects this year alone totaling over $170k. These apps should have never been on MongoDB in the first place. Total mess that cost them lots of $$. I am okay if others continue to use MongoDB. It will keep my team gainfully employed ;)

icc97 10y ago
Wow, that is truly impressive. I'd be surprised if there's any examples of people prepared to spend the same amount of money to go the opposite way.
- 20years 10y ago
  To be fair, the projects pretty much required a full code re-write too. MongoDB in both cases was one of many bad decisions. Choosing all hipster stuff is what got them into trouble.
pbreit 10y ago
If you're gonna migrate to SQL, wouldn't Postgres be a better call?
- 20years 10y ago
  Postgres is great and was on the table but MySQL was chosen for a variety of reasons. In particular, one of the companies already had a lot of other things running on MySQL that they wanted integrated. They were happy with that setup and MySQL was a perfectly fine choice for the app we were migrating from MongoDB.

yongjik 10y ago

> Here we have copies of user data inlined. This is Joe’s stream, and it has a copy of his user data, including his name and URL, at the top level. His stream, just underneath, contains Jane’s post. Joe has liked Jane’s post, so under likes for Jane’s post, we have a separate copy of Joe’s data. > You can see why this is attractive: all the data you need is already located where you need it. That doesn't sound attractive at all. That sounds more like a recipe for "we use five hundred times more RAM than the deduplicated data that we actually have" type of disaster. ...All for a production data set which could be "turned into about 1.2 million rows in MySQL." I think the moral is "If you think your data is too big for a relational database, you're almost certainly wrong."

GickRimes 10y ago

"Shards are the secret ingredient in the webscale sauce, they just work.." :) https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs

nnain 10y ago

However, MongoDB seems to be marching on fine in comparison to other popular NoSQL DBs (viz. CouchDB and RethinkDB) for the past decade - https://www.google.com/trends/explore?date=2006-08-16%202016-08-16&q=MongoDB,CouchDB,RethinkDB https://www.google.com/trends/explore?date=2006-08-16%202016...

danjc 10y ago

Non-relational database for a million rows of data? Madness.