19 ms·

Why You Should Never Use MongoDB

• 3327 13y ago
  This is a well known and well document "down side" of mongo. Frankly the analysis of your article is jeopardized by your first line stating, "I am not a database designer". Mongo has its downsides that are well known, but there are also very good reasons to use Mongodb too. although its a lengthy article with good examples it states nothing more than an obvious caveat mongo has which is well known and documented.
• edude03 13y ago
  I hate link bait like this. The real title should be "Why you should never use a tool without investigating it's intended use case"
  • Robin_Message 13y ago
    But the point is that there is no use case. Relational databases and normalisation didn't arise because a load of neckbeards wanted bad performance and extra complexity. The point of the article is that the world is relational, and because Mongo isn't, it'll bite you in the ass eventual. Sure, that's a specialisation of what you said, but still a useful one, as it allows you to immediately know you shouldn't use Mongo (unless your data is all truly non-relational, and you know you'll never integrate it with any relational data, which, without a crystal ball, you can't know, so don't use it.)
    • btown 13y ago
      As soon as you have users, you'll want to handle relationships between users, whether that's outward-facing or for internal analytics. All products have users by definition. Therefore...
    • exelius 13y ago
      There is a use case, but internet hype has gotten everyone wanting to use Mongo when there's no real reason to. Postgres scales nearly as well as Mongo while being a lot more flexible. That said, Mongo has some real benefits for non-relational computing (see mapreduce) that could make some of the abstraction headaches and lack of data model flexibility worth it for very large data sets. But I sort of agree; Mongo tends to be overused by startups who are trying to solve a scalability / performance problem before they have one. In the process they often end up running into data model limitations because stuff moves fast early on and you can't foresee what you'll need in a year.
    • twic 13y ago
      Yes. There seem to be a lot of people with quite poor reading comprehension commenting here. The case that the article makes is something like: 1. Document stores are no good for data with non-strictly-hierarchical structure 2. All interesting data has some non-strictly-hierarchical structure The first point is common knowledge nowadays. It's really the second point that is interesting. Moreover, interesting and correct.
• mkohlmyr 13y ago
  .. For relational data. Is what the title should be. But that is far less effective linkbait.
• jliptzin 13y ago
  It's certainly not a one size fits all solution, but saying NEVER use it for anything is a little strong. I've been using it in production now for 6 months on a fairly well trafficked site without a single disruption that wasn't the fault of application layer code.
• colinbartlett 13y ago
  This is ridiculous linkbait bullshit. Anyone who dismisses document stores entirely has lost all my respect. It wasn't the right solution for your problem, but it might be the right solution for many others.
  • neoveller 13y ago
    Agreed. Also, this looks way more like a case where the author mis-structured his data for his intended use case, and is blaming the tool instead of the skill level used to implement it. Nesting deeper than one level in a document is rarely going to result in sufficient query capability with respect to the nested items. Even MySQL can't nest rows inside other rows, which is what he seems to have wanted. Maybe he chose MongoDB because he wasn't ready to think around the architecting issues that an SQL-based database would require, which happen to be, although not immediately obvious, similar to those in Mongo.
    • mossity 13y ago
      Despite being a programmer, I believe Sarah is a woman.
      • pearjuice 13y ago
        The more reason this article should be taken with a grain of salt.
        • DeadSuperHero 13y ago
          Seriously, dude? It's attitudes like that which makes women hesitant to become coders and computer engineers. It doesn't matter what's between a person's legs; just that a person can code, enjoys doing it, and knows what they're talking about.
          • pearjuice 13y ago
            For years men were dominant in computer science and engineering and I really see no reason why this should change. Only in recent time with this whole third wave of post-feminism certain groups of females think it is their task to overthrow men supremacy in said fields. They are way too emotional for this profession and this results in drama and absolute shit code in production. I am not saying the author of the article submitted has fallen victim to described wrong doings, I am only saying that at all cost unknown teritorry should be approached with extreme care. You don't know what is subliminally hidden until you realize. Too late.
• exelius 13y ago
  +1 on all the linkbait comments. MongoDB was not a good fit for this project; but there are a great many projects where MongoDB is a great fit. If you were dumb enough to just start using MongoDB (or mySQL, or whatever) without matching it to your data model, that's on you.
• Axsuul 13y ago
  Can anyone explain what are some actual real-life good uses for MongoDB?
  • boomzilla 13y ago
    Alright, so I am not the only one who's been very curious too. Please, someone writes up a real use case where MongoDB is used as the only/main data store, and not a persistent cache in front of relational database? Thanks.
    • nevi-me 13y ago
      TL:DR; Mongo works for me ('us'), and when I get some time I'll write a post on what we're doing with it, and why it works for us. I am ('we are') using Mongo for a public transit planner in South Africa. It's not yet production-ready, but beta testing is going well. Let me paint the picture before I go on to justify our use of Mongo. In South Africa there are trains, buses, minibus and other services (metered taxis, shuttles). Trains run on stop-by-stop schedules, all of the bus services just a normal departure-based schedule, and minibus completely dynamic. In order to implement a well-organised integrated planner, you have to view all of these as one 'type' of service. There's then how pricing is calculated for each of the services. There are many different ways in which pricing is calculated, being: (distance-based, zone-based, pricing matrix, fixed minimum with variable charge etc.), then there's ticketing, discounts etc. That too we needed to represent in a simple structure. Now, my SQL is pretty good, I don't frown upon indices nor joins, but what I can say is that in my initial implementation of the whole idea, I faced a number of problems, being: (1) what level of normalisation/denormalisation is necessary? (i.e. what should I join, what should I keep in the same table) (2) I'm essentially using a graph, except that nodes aren't always connected, so how do I traverse the graph when it's actually 'broken'? (excuse me if I get the terms wrong, I'm actually a technical accountant, programming is my second love) (3) I'm working with location data, I can't expect to use the Haversine formula or equivalents, how can I index both locations, and routes? How do I even store routes? (blob of serialised arrays?) (4) How do I reduce development time, to reduce the amount of time I spend refactoring schemas/code when I want to implement some new 'shiny' feature? These were my main concerns, as they were the problems that I had with MySQL. PostgreSQL would have done a good job for (1) and (3), and maybe (2), but I was still concerned with (4) as working with PHP/MySQL isn't the friendliest of things. Doing 'in()' queries is one example, as I have to parse an array to a string before using an in() function. Mongo initially appealed to me because it was marketed as 'schemaless', but even someone with little knowledge as I knew that it should be taken with a grain of salt. The benefit here was that I could store different services with different attributes in one collection. If a service is a minibus, I add all the fields that I need for the minibus, and omit the ones for a train for example. Similarly with the pricing structures. At first it was difficult grasping the 'store everything in one document' model, but I got the hang of it, and now my schema is 'frozen', so (1) has been solved. I don't use joins or any simulation thereof. Yes, because storing I store what I need in one document, I don't need to go back to Mongo to find joining data. (2) A graph database wouldn't work well for me, because even though services link together, there are instances where the commuter will have to walk to join another service. How do I do that? I initially created manual walking links in MySQL but that was naive and stupid (trying to avoid Haversine). Obviously PostgreSQL distance-based queries would also work here. Another thing against a graph database is that my project doesn't just rely on traversing graphs all day, there are other things which I need to do, like analytics. (3) To be honest, even though I'm confident with my SQL knowledge, all the PostgreS/PostGIS functions felt a bit intimidating at first. I can't afford a $250k/year DBA, so I have to know what I'm doing on the database as well as the client/server. I find learning how to use Mongo to be easy, even though people say their query 'language' is (insert bad word), I find it quite user-friendly. Mongo's geospatial support, and GeoJSON as of 2.4, made things like storing a route, and running typical queries on it, very easy. One other thing that we discount is the transparency of data in Mongo, yes they should use field compression and other space-saving techniques, but until them, I see the benefit in being able to do a db.collection.find() and being able to look at the whole document without having to join any tables (it's just a bit quicker I guess). Which brings me to (4): the most fun thing that I had to implement was scheduling, bearing in mind that there are scheduled services, and variable/random ones. Let's say I have a separate table for schedules, and I want to find: - x services, - that start/pass at [lat,lon] - which are operating during ab:cd AM - which have the next schedule/estimate within m and n. Sure, you could do it with a JOIN, but why do that when you can do it all from a single document? Lastly, experienced programmers tend to take for granted the benefit of simplifying certain things for novices/beginners. The reason why JSON is taking over, besides that it's a more compact and readable expression than XML, is that it's also easy to work with. Why should I worry about converting associative arrays to strings in order to do an in()? Instead of saying something like "in(array)" directly? Even though I was forward thinking in my schema design, there were a few changes along the road when I realised that something wasn't working. Making a change in the schema was quick, and I didn't have to spend a lot of time making sure that my data is still fine. Please note that I didn't talk about 'web scale, speed' or all those other things. Under my current hardware I would need to cover 3 countries' transit systems before I need to shard. I am running a single-node replica so I can enable backups. That's just my view, I wrote this in pieces, so I might appear to be all over the place. I'll write a thought-out post detailing why Mongo is currently working for us/me.
• lafar6502 13y ago
  Oh, God, an advisory article by people who have just learned what a relational db is. What's next?
• kylemaxwell 13y ago
  "You should never use MongoDB [the way we did]". For some use cases, it would be a terrible decision, as this project learned. In others, it works fine, even with the "eventually consistent" sort of thing. I knew it would go bad when the author immediately started talking about web apps from the get-go, because (as unusual as this may seem to some subset of developers) not everything is a webapp.
• danso 13y ago
  My advice to the OP: Re-jigger this article and retitle it: "The Data-Design of Social Networks". That would be a worthwhile read and I appreciate the detail that the OP goes into. One of the subheads should be: "Why we picked the wrong data store and how we recovered from it" And not to be snarky about it, but an alternative title is: Why Diaspora failed: because a Ruby on Rails programmer read an Etsy blog and thought they understood databases
• chaz 13y ago
  Linkbait title aside, it's actually a helpful example for directing a database novice on when to not use a document store. I could have used this post a few times in the past few years.
  • rch 13y ago
    Agreed. Despite the unfortunate title, this is an informative, well written and entertaining article that I might refer to in the future. It would be better if there was a followup on when it would in fact be appropriate to introduce a document store to a project.
    • danso 13y ago
      I agree that the OP is lengthy, and putting together this well-illustrated post is no easy feat. However, I don't think the OP should be the one to write about when you should use a document store. Maybe I'm too annoyed by the poorly chosen title. Or that I read that entire post and was thinking where's the punchline? On one hand, I credit the author for thinking things through. On the other, the fact that she unequivocally attributes this issue to MongoDB shows that she currently lacks the domain knowledge to consider appropriate use cases. It's not a MongoDB problem, it's a problem inherent to this data structure, and someone more well-versed in this topic would not conflate the issue...just as a decent IT person would not blame "Windoze" for the fact that she can't get good Wifi reception in the office. OK, to be even more petty...I think what really aggravates me is how the OP says she's not a database expert -- which is a good disclosure, but self-evident -- but attempts to assert authority by saying "I build web applications...I build a lot of web applications"...Uh, OK, so what you're saying is that it's possible to be an experienced web developer and yet be a novice at data design? If that was the angle of the OP, I'd give it five stars. Such sentiment cannot be overstated.
      • rch 13y ago
        Well you're right of course that web developers (and business analysts, and politicians, etc.) can absolutely get by for a staggeringly long time with novice-level abilities. That problem is only getting worse as the tools get better. Luckily I don't have to judge the OP on that basis since that's what markets are for. And maybe someone else, who has tackled enough difficult problems over time to evolve a nuanced and technically informed opinion of various data modeling and management options, should write the response I mentioned. I'd argue there are plenty examples of that material available already. The OP, on the other hand, would be writing from the perspective of a professional user who might choose a tool off the shelf at the recommendation of a colleague, and whack it against the problem du jour to see if it works or not. This is a common enough approach that there is at least a chance that a followup would have some value. I can't really expect everyone who makes a living writing web applications to understand CS fundamentals, any more than I would expect it from chemical engineers or physicians. It is nice to be able to point representative members of that audience to an article that resonates with them, and not have to try to translate my opinions into similar language (with or without cat gifs). Edit: I actually think Journeyman would be a more appropriate term than novice.
• gress 13y ago
  Many comments attacking the author of original article, but not a single one addressing the arguments.
  • bronson 13y ago
    That's because her arguments don't make a lot of sense. It's like titling a blog post "Never use a hammer!" and follow it with line after line of "I'm not a refrigerator repairman but my refrigerator uses Phillips screws so I used a hammer to remove them and it kind of worked but it took far long to do the job and damaged some of the screws..." Short of "use the right tool for the job" (which a lot of people here have already said), what do you expect? Mongo is a document store. If she'd used it to store documents then I think her blog post would have been quite positive.
    • nostrademons 13y ago
      Her point was that things that often look like documents on the surface (eg. activity streams, TV show listings) are anything but once you dig into the problem area. It's worth remembering that the next time you're faced with something that looks like a document.
    • gress 13y ago
      If you're right, it would be helpful to point to an unambiguous description of what constitutes the kind of 'document' that mongo is suited for, and what does not.
      • bronson 13y ago
        That's like asking for an unambiguous description of what makes a language good. Even if I were to write a book it would be oversimplified and unsatisfying. SO, OK, here's the oversimplified version... basically, if you have large amorphous chunks of data, easily denormalized so very few joins required for typical queries, then you have a document-friendly dataset. Medical records, court docket entries, things where a SQL representation has many tens of often-NULL columns and only a few foreign keys, those are probably good document-based data. What Sarah describes is exactly the opposite: small, deeply nested, tightly joined, difficult to denormalize, nuggets of data. Expecting a document-oriented database to handle this is like expecting a SQL database to handle complex graph data: it can be done, but it's slow and the workarounds are not pretty. It doesn't make a lot of sense to complain about that. Hope that makes sense.
• ukd1 13y ago
  This should read one of; 1. why you shouldn't use things you don't understand. 2. I wish I'd rtfm with Mongo 3. schema design for mongodb, what I wish I'd known
• raverbashing 13y ago
  Really, in some places it hurts * We stored each show as a document in MongoDB containing all of its nested information, including cast member* I've seen this in people using MongoDB and the bough the BS that because "it's a document store" there should be no link between documents. People leave their brain at the door, swallow "best practices" without questioning and when it bites them then suddenly it's the fault of technology. " or using references and doing joins in your application code (double ugh), when you have links between documents" 1) MongoDB offers MapReduce so you can join things inside the DB. 2) What's the problem to have links between documents? Really? Looks like another case of "best practice BS" to me
  • liveoneggs 13y ago
    at mongo training we were told map/reduce did not offer performance and to avoid it for online use. You must use the "aggregation framework".
  • lhc- 13y ago
    Links in mongo aren't really links though; its up to the application to handle the "joins", which really means making an extra query for every linked item. It's like SQL joins except without any of the supporting tools or optimizations that exist in RBDMS.
    • raverbashing 13y ago
      Yes, it is manual But you can query for a list of ids for example, using the 'in' operator and a list. http://docs.mongodb.org/manual/reference/method/db.collection.find/#db.collection.find http://docs.mongodb.org/manual/reference/method/db.collectio...
      • SigmundA 13y ago
        Isn't this done client side? Without joins in the db engine itself locality is much worse along with lost opportunities for optimization leading to much worse performance.
• deleted 13y ago
  [deleted]
• JangoSteve 13y ago
  As others have pointed out, this article can basically be summarized as, "don't use MongoDB for data that is largely relational in nature." Mongo (or most document stores) are good for data that is naturally nested and silo'd. An example would be an app where each user account is storing and accessing only their own data. E.g. something like a todo list, or a note-taking app, would be examples where Mongo may be beneficial to use. A distributed social network, I would have assumed, would be the antithesis of the intended use-case for a document store. I would have to imagine a distributed social network's data would be almost entirely relational. This is what relational databases are for.
  • raverbashing 13y ago
    Or maybe something like this: A Graph Database http://en.wikipedia.org/wiki/Graph_database http://en.wikipedia.org/wiki/Graph_database
    • mhluongo 13y ago
      Yep, the "graph databases are too niche to be put into production" bit urked me- Neo4j et al are in plenty of large production systems. OTOH maybe, due to the distributed nature of the project, they didn't want to distribute a less-known database?
      • iSnow 13y ago
        I guess in 2010, Neo4j did not have that much exposure as they have today. Still I concur that the author should not brush graph databases aside for something like a social network - they seem a better solution than a RDBMS.
        • twic 13y ago
          I suspect there isn't actually a lot of need for graph operations in a social network. At least, not in implementing features for the users. A distinctive thing about social networks is that although the users form a network, they are primarily social - they're interacting with their friends. They will end up interacting with friends of friends via their friends (eg having a flamewar with your cousin's neighbour in the comments of your cousin's post about potatoes), but not with friends of friends of friends or any degree of separation further out. The queries needed are overwhelmingly local, and a boring old relational database will handle them fine. Where a graph database might shine is in analytics over the whole network, looking for trends, hubs, clusters, etc, lthough i'm not convinced it would be any better than a relational database which supports recursive queries (as PostgreSQL does). However, this is exactly the sort of privacy-busting awfulness that Diaspora was built to escape from!
• kcorbitt 13y ago
  I've seen a fair number of articles over the last couple of years comparing the strengths and weaknesses relational/document-store/graph databases. What I've never seen adequately addressed is why that tradeoff even has to exist. Is there some fundamental axiom like the CAP theorem explaining why a database like MongoDB couldn't implement foreign keys and indexing, or why an SQL couldn't implement document storage to go along with its relational goodness? In fact, as far as I can tell (never having used it), Postgres's Hstore appears to offer the same advantages as a document store, without sacrificing the ability to add relations when necessary. Where's the downside?
  • boomzilla 13y ago
    But it's not web scaled :) http://www.youtube.com/watch?v=URJeuxI7kHo http://www.youtube.com/watch?v=URJeuxI7kHo
  • ddebernardy 13y ago
    > why an SQL couldn't implement document storage to go along with its relational goodness? (…) Postgres's Hstore appears to offer the same advantages as a document store, without sacrificing the ability to add relations when necessary. Where's the downside? PostgreSQL can store arbitrary unstructured documents just fine: hstore, json, … Each come with the possibility to actually index arbitrary fields within the documents using a BTREE index on an expression, and arbitrary documents wholesale using GIST index. Besides the need to know a thing or two on query optimization, the only downside I can think of is that ORMs are usually broken (Ruby's Sequel is a notable exception). But this isn't a problem with Postgres itself; it's a problem with ORMs (and training, admittedly).
  • mhluongo 13y ago
    Typically as your data model complexity ("relatedness") increases, it's more difficult to scale. I'm not sure about anything like CAP, but I do know that in graph-database land we have to remind ourselves that general graph partitioning is NP-Hard, and that our solutions will need to be domain-specific.
• meritt 13y ago
  OP should have been using a graph database. Ranting about MongoDB because it doesn't support what it's not designed to support is a bit silly. A RDBMS would have been just as poor of a choice here.
  • ddebernardy 13y ago
    Facebook seems to be doing quite fine by combining SQL and Memcached.
    • weixiyen 13y ago
      They probably don't use relational databases the way you do for smaller projects that don't need to scale to the millions.
• johnymontana 13y ago
  The article mentions that they considered a graph database, but considered it too niche for production. Is that the general opinion on graph databases (like neo4j) at this point? Not production ready? This project seems like a perfect application for a graph database.
• tayzco 13y ago
  This is just a naive article from a naive developer who shouldn't be responsible for choosing data stores in any project. 4-6 projects is a lot? I think you need to get your head out of your a (like most ruby devs ;).
  • kul_ 13y ago
    we dont treat well the kind who write 1000 lines of codes for adding two integers over here mister.
• buckbova 13y ago
  Not sure why it has to be one or the other. It seems perfectly reasonable to store the users profile information and media in mongodb and your likes and comments in a relational db.
• electrichead 13y ago
  I'm going to reiterate as others have done - this is an area where a good graph database would blow all the others out of the water. I am currently using neo4j for a web app and find it to be extremely good in terms of performance. There is really only one downside to using a graph database - they are not really scalable horizontally as you might want. They need a fair bit of resources. But in terms of querying, they would be unparalleled in this particular use-case. They are also not in infancy - they are in use in many places where you wouldn't expect them and which aren't discussed. One big area is network management - at least one major telecom uses a particular graph db to manage nodes in real-time.
  • mason55 13y ago
    > they are not really scalable horizontally as you might want Seems like this would be a huge drawback for a project whose entire raison d'etre is horizontal scaling.
• hkarthik 13y ago
  >> Some folks say graph databases are more natural, but I’m not going to cover those here, since graph databases are too niche to be put into production. Is this really true? It sounds like both relational DBs and document DBs are a poor choice for the social network problem posed. I've actually dealt with this exact problem at my last job when we started on Mongo, went to Postgres, and ultimately realized we traded one set of problems for another. I'd love to see a response blog post from a Graph DB expert that can break down the problem so that laymen like myself can understand the implementation.
  • yid 13y ago
    > Is this really true? Facebook's TAO is a giant graph database built on top of MySQL [1]. I'd say it's pretty production-ready, because Facebook's social graph probably has at least hundreds of vertices. [1] https://www.facebook.com/notes/facebook-engineering/tao-the-power-of-the-graph/10151525983993920 https://www.facebook.com/notes/facebook-engineering/tao-the-...
    • ris 13y ago
      There's a difference between ready-for-production and ready-for-production-if-you-have-the-entire-team-of-developers-that-wrote-it-on-hand-all-the-time.
  • the1 13y ago
    what's wrong with symlinks on transactional filesystem?
    • jahewson 13y ago
      Millions of small files is the worst workload for pretty much every filesystem. Data locality and fragmentation can end up becoming real problems too.
  • fat0wl 13y ago
    I would look at Neo4j. I originally came across it when vetting Grails (it has a Grails plug-in) and it seems to be one of the heavy contenders in terms of a production-ready graph DB. People (this article's included) seem to say that production-ready graph DBs don't exist. Maybe these projects are still trying to gain traction? I expect some stable builds will be out there soon if they aren't already... http://www.neo4j.org/ http://www.neo4j.org/
• marvwhere 13y ago
  TL;DR would be nice =/ all in all i do not see all problems u see, we running mongo with elastic search on 30k unique page per day, and we do not have big problems.
• weddpros 13y ago
  The problem described here is about the lack of transactions only... The relational nature of pgsql or mysql is not what makes a difference when you want "all or nothing" insertions or updates...
• neokya 13y ago
  Though title sounds like a link bait, this is actually eye opening article for database layman like me. Very clearly written. Now, what is MongoDB fit for? Most of the web applications are what author gives example, complex and having inter-relationships. Can someone light up?
  • jacques_chester 13y ago
    Like the article says, it can be suitable as a caching layer in front of a DB, especially for web apps that deal in ephemeral JSON documents.
• seivan 13y ago
  These days I tend to use SQL for data, and Redis for complex relationships and graphs. It's good enough for me to combine those. Usually the relationships are stored with ids on ordered sets. And I just do an id lookup on postgresql.
• Justsignedup 13y ago
  SQL is actaully a rediculously elegant language at expressing data, and relationships. NOT a good general purpose language. So I tend to always favor sql for relational data, actually most data. queues, caches, etc = nosql solution. They tend to have much more features around performance to handle the needs of these problems, but not much in terms of relational data. If you study relational databases and what they do, you will quickly find the insane amount of work done by the optimizer and the data joiner. That work is not trivial to replicate even on a specific problem, and ridiculously hard to generalize. And so this article's assertion that mongodb is an excellent caching engine, but a poor data store is very accurate in my eyes.
  • _sh 13y ago
    No. SQL is actually pretty third-rate at expressing data and relationships. My preferred way of expressing data and relationships is the programming language I am writing in. The problem with SQL is that it is not an API, it's a DSL. Which usually means source-code-in-source-code, string concatenation/injection attacks, and crappy type translations ('I want to store a double, what column type should I use? FLOAT? NUMERIC(16,8)?'). Even as a DSL it's pretty low-brow: just look at how vastly different the syntax is between insert and update, or 'IS NULL'. For all those who love SQL, consider having to address your filesystem with it. Directories are tables, foreign-keyed to their parent, files are rows. There's a good reason why this nightmare isn't real: APIs are preferred over DSLs for this use case. And so too for databases, because they are the same abstraction. Don't get me wrong, I love relational algebra and the Codd model, but SQL just aint it. SQL has survived because of its one and only strength: cross-platform. And like all cross-platform technologies, such as Java bytecode and Javascript, its rightful place is a compilation target for saner, richer, more expressive technologies. This is why I always use an ORM and have vowed to never, ever, write a single line of SQL again.
    • spion 13y ago
      How about hybrid sql-builder / data grouper solution? Not limited to ORM methods - get the full power of SQL instead. Not string concatenation - get the full power of the language to build queries. Also, the ability to get join results in either flat or grouped form. For example https://github.com/doxout/anydb-sql https://github.com/doxout/anydb-sql (shameless plug)
• scbrg 13y ago
  I must have read a dozen (conservative estimate) articles now all called "Why you should never use MongoDB ever" - or permutation thereof. Each and every one of them ought to have been called "I knew fuckall about MongoDB and started writing my software as if it was a full ACID compliant RDBMS and it bit me." There are essentially two points that always come up: 1. Oh my God it's not relational! Well, you could argue that if you move from a type of software that is specifically called RELATIONAL Database Management System to one that isn't, one of the things you may lose is relation handling. Do your homework and deal with it. 2. Oh my God it doesn't have transactions! This is, arguably, slightly less obvious, and in combination with #1 can cause issues. There are practices to work around it, but it is hardly to be considered a surprise. I keep stumbling on these stories - but still these are the two major issues that are raised. I'm starting to get a bit puzzled by the fact that these things are still considered surprises. In either case, I'm happily using MongoDB. It has its fair share of quirks and limitations, but it also has its advantages. Learn about the advantages and disadvantages, and try to avoid locking too large parts of your code to the storage backend and you'll be fine. FWIW, I think the real benefit of MongoDB is flexibility w/r to schema and datamodel changes. It fits very, very well with a development process which is based on refactoring and minor redesigns when new requirements are defined. I much prefer that over the "guess everything three years in advance" model, and MongoDB has served us well in that respect.
  • jahewson 13y ago
    I'm not sure what these two strawmen have to do with the article. Perhaps you've been reading another article?
  • rdtsc 13y ago
    > I must have read a dozen (conservative estimate) articles now all called "Why you should never use MongoDB ever" Strange statistical oddity if you ask me right? How many "don't use PostgreSQL" or "don't use Cassandra" or "don't use SQLite" have you seen? Not as many. It is just very odd isn't it... So either everyone is crazy or maybe there is something to it. I lean towards the later here. > 1. Oh my God it's not relational! ... > 2. Oh my God it doesn't have transactions! Maybe those, you forget about: 3. Claim "webscale" performance while having a database wide write lock. 4. Until 2 years ago shipped with unacknowledged writes as a default. Talk about craziness. A _data_base product shipping with unacknowledged send-and-pray protocol as a default option. Are you surprised people criticize MongoDB? Because I am not at all. Sorry but after that I cannot let them within 100 feet of my data. They are cool guys perhaps and maybe having beers with them would be fun, but trusting them with data? -- sorry, can't do.
• ecaron 13y ago
  The best thing about this article is it demonstrates the problem with pg's "The submission must match the title" policy.
• crystaln 13y ago
  There are very few use cases that can be solved with mongo that can't be solved equally well using nosql inspired features of Postgres and other simple tools. Data often has relational characteristics we don't anticipate. That's why we build flexible schemas. Mongo schemas have none of that of flexibility.
• cnp 13y ago
  What a FANTASTIC write up! So much clarity here.
• asdasf 13y ago
  >Seven-table joins. Ugh. Where does this attitude come from in the first place? Even when I was just first learning SQL the notion of doing multiple joins was never off-putting or scary. Quite the contrary, the fact that joining two relations produces a relation which I can then use in more joins seemed like a perfectly elegant abstraction. >On my laptop, PostgreSQL takes about a minute to get denormalized data for 12,000 episodes http://www.postgresql.org/docs/9.3/static/sql-createindex.html http://www.postgresql.org/docs/9.3/static/sql-createindex.ht...
  • samspot 13y ago
    It's because so many of us have seen joins go bad, either due to poor code or other factors. I regularly hear people talk of never doing joins again. Not that I agree with this attitude, but it is certainly out there.
    • asdasf 13y ago
      I've been using relational databases for 15 years and I've never "seen joins go bad". I don't even know what that means. I know some people avoid joins, but from my interactions with those people it has always been because they heard the myth that joins are slow, and foolishly figure writing a naive join implementation in PHP will be faster.
  • smsm42 13y ago
    Joins are potentially explosive (cartesian explosion :). So they have to be treated with respect, just as other explosive substances should be. That doesn't mean they should never be used - modern industries use thousands of dangerous things routinely. But proper handling is necessary - indexes, watching the data consistency, etc. So it is a good instinct to feel uneasy when there are seven joins around. It's not necessary fatal but it is something that would require attention.
    • asdasf 13y ago
      I've never written an application that didn't involve multiple queries with 7+ joins. There is absolutely no reason that should make anyone feel uneasy, it is perfectly normal and expected.
• deleted 13y ago
  [deleted]
• halayli 13y ago
  Don't blame it on MongoDB if you are using the wrong tool for the job.
  • jonknee 13y ago
    The point of the article was that there isn't a job that MongoDB is the tool for. (See the last example where the model fit perfectly until a feature was needed that blew everything up.)
    • ruok0101 13y ago
      Wasn't the point actually that even in that second example, they didn't have the foresight to ask the client about the need to cross reference tv shows ahead of time? And for the record, how on earth is this MongoDB specific? Almost all NoSQL solutions fall into this same situation. I was at least expecting more complaints about the early days of mongo's writes not being durable... At least that argument has merit.
  • pekk 13y ago
    If MongoDB promotes itself for the wrong jobs, then the shoe fits.
• exclusiv 13y ago
  Do NOT use MongoDB unless you understand it and how your data will be queried. Joins like the author mentions by ID is not a bad thing. If you aren't sure how you are going to query your data, then go with SQL. With a schemaless store like Mongo, I've found you actually have to think a LOT more about how you will be retrieving your information before you write any code. SQL can save your ass because it is so flexible. You can have a shitty schema and make it work in the short term until you fix the problem. I wrote many interactive social apps (fantasy game apps) on Facebook and it worked incredibly well and this was before MongoDB added a lot of things like the aggregation framework. The speed of development with MongoDB is remarkable. The replica sets are awesome and admin is cake. It sounds like the author chose it without thinking about their data and querying upfront. I can understand the frustration but it wasn't MongoDB's fault. This is a big deal for MongoDB: https://jira.mongodb.org/browse/SERVER-142 https://jira.mongodb.org/browse/SERVER-142. Let's say you have comments embedded on a document and you want to query a collection for matches based on a filter. If you do that, you'll get all of the embedded comments back for each match and then have to filter on the client. IMO, when the feature above is added, MongoDB will become more usable for more use cases that web developers see.
• andrelaszlo 13y ago
  I'm not arguing against this article, they seem to have made some poor choices along the road, but to say "you should never use MongoDB" is silly - even if you add "...for relational data". It has it's obvious drawbacks, of course, but MongoDB is way more than just a caching layer. Here's some social networks that are running MongoDB: http://www.mongodb.org/about/production-deployments/#social-networking http://www.mongodb.org/about/production-deployments/#social-... The list includes Foursquare. They have been running Mongo as their main storage engine for about four years now. They migrated away from MySQL, as discussed in this video: http://youtu.be/GBauy0o-Wzs?t=2m30s http://youtu.be/GBauy0o-Wzs?t=2m30s
• m_mueller 13y ago
  I don't know much about MongoDB, but I've been using a lot of CouchDB for my current project. Am I correctly assuming that MongoDB has no equivalent for CouchDB views? Because if it had, all these scenarios shouldn't be a problem. Here's how relational lookups are efficiently solved in CouchDB: - You create a view that maps all relational keys contained in a document, keyed by the document's id. - Say you have a bunch of documents to look up, since you need to display a list of them. You first query the relational view with all the ids at once and you get back a list of relational keys. Then you query the '_all' view with those relational keys at once and you get a collection of all related documents - all pretty quickly, since you never need to scan over anything (couchDB basically enforces this by having almost no features that will require a scan). - If you have multiple levels of relations (analogous to multiple joins in RDBMs), just extract they keys from above document collection and repeat the first two steps, updating the final collection. You therefore need two view lookups per relational level. All this can be done in RDBMs with less code, but what I like about Couch is how it forces me to create the correct indexes and therefore be fast. However, if my assumption about MongoDB is correct, I have to ask why so many people seem to be using it - it would obviously only be suitable in edge cases.
  • rdtsc 13y ago
    Also CouchDB has better safety. Its append-only files allows you to make hot backups and safely pull the plug on your server if need be without worrying corrupting data. Plus change feeds and peer to peer replication are first class citizens in the CouchDB. Once you start having large number of clients needing realtime updates, having to periodically poll for data updates can get very expensive.
    • m_mueller 13y ago
      Offline capable peer to peer replication was the main reason I chose CouchDB - we needed something that would realistically run on clients, even mobile devices. NoSQL we mainly chose because we needed schema-less data (the whole system relies on ad-hoc design updates). It's basically an information system IDE with rapid application development.
    • 13y ago
• gmjoe 13y ago
  > Seven-table joins. Ugh. What? That's what relationship databases are for. And seven is nothing. Properly indexed, that's probably super-super-fast. This is the equivalent of a C programmer saying "dereferencing a pointer, ugh". Or a PHP programmer saying "associative arrays, ugh". I think this attitude comes from a similar place as JavaScript-hate. A lot of people have to write JavaScript, but aren't good at JavaScript, so they don't take time to learn the language, and then when it doesn't do what they expect or fit their preconceived notions, they blame it for being a crappy language, when it's really just their own lack of investment. Likewise, I'm amazed at people who hate relational databases or joins, because they never bothered to learn SQL and how indexes work and how joins work, discover that their badly-written query is slow and CPU-hogging, and then blame relational databases, when it's really just their own lack of experience. Joins are good, people. They're the whole point of relational databases. But they're like pointers -- very powerful, but you need to use them properly. (Their only negative is that they don't scale beyond a single database server, but given database server capabilities these days, you'll be very lucky to ever run into this limitation, for most products.)
  • pekk 13y ago
    When program errors pass silently, that is a legitimate problem in the toolchain.
  • achy 13y ago
    I agree with you to a point. Joins are your friend. But trying to pull out all of the information about a graph of 'objects' using a single query with multiple one-to-many and many-to-many joins is just as foolish in SQL as in Mongo.
  • mkoryak 13y ago
    This article ends up agreeing with you at the end, by the way.
  • _pferreir_ 13y ago
    > A lot of people have to write JavaScript, but aren't good at JavaScript, [...] they blame it for being a crappy language, when it's really just their own lack of investment. I think it's pretty much an accepted fact that JS has its problems. Even Brendan Eich has been quoted as admitting it. (Note: I am a JS developer myself)
• No1 13y ago
  The whole post hinges on the statement: "When MongoDB is all you have, it’s a cache with no backing store behind it. It will become inconsistent. Not eventually consistent — just plain, flat-out inconsistent, for all time." OK, here is a chance to share a little insight. At what point did Mongo become hopelessly inconsistent? Were you ever able to determine why? Why bother with cute pictures and verbose explanations of simple schemas when the conclusion is just that Mongo breaks no matter what without further explanation?
  • gcv 13y ago
    Probably because the project was forced to keep copies of data all over the place, and was distributed to boot. There was probably a reasonable bug or edge case which caused some copies to conflict with each other, and since writes are destructive, it became impossible to reconcile the conflicts.
  • MoosePlissken 13y ago
    I thought this statement was confusing as well. I can see how the given schema could cause problems with inconsistency, but there seem to be a few logical steps missing in the jump from that to "Mongo will never be consistent and we need to migrate to mysql".
  • regularfry 13y ago
    That's not a property of Mongo, it's a property of any non-trivial caching layer that you can't regenerate. Something will go wrong, and then you're stuffed.
• wehadfun 13y ago
  This was the best explanation of MongoDB I've read.
• bsaul 13y ago
  The most interesting part is to make a difference between your primary data store and your secondari(es) ones. For "friends of friends" queries you could definitely use graph dbs, for "tree hierarchical" data then a document based is good. Secondary dbs are chosen depending on queries. For analytics then choose columnar dbs or map/reduce friendly data stores, etc... But primary db needs powerful querying possibilities, strong consistency and durability,as well as really good admin toolings. That's the "fall back", to be used as a last resort, but that's also the point of truth of your system. Then comes optimizations, and pipe-like data processing to move data from primary to secondary dbs ( or to both in parallel). That's why i've never been found of using new technologies such as mongodb or couchdb or anything like that for my primary db.
• petepete 13y ago
  The 'main' app at my place of work uses MongoDB this way; it even implements a 'relationships' collection that is used for one-to-many 'joins' (ugh, just thinking about it makes me feel ill). Unfortunately, I joined when the initial write was nearing completion and was unable to steer the team in the right direction in time. I just submitted a proposal for a ground-up rewrite; unless it's accepted I'll be leaving promptly.
  • jacques_chester 13y ago
    In fairness to MongoDB, I've seen this done in the relational world. I once worked at a place who had a schema where everything joined through a single table -- "TableRow_TableRow", which had six fields. Two IDs (which were varchars) and metadata_1 through metadata_4 (also varchars). They couldn't understand why it was so slow. But hey, it's super flexible, right?
• just2n 13y ago
  I once tried to insert a screw using a hammer. I'll be writing my article "Why You Should Never Use A Hammer" shortly. And here's the crux of the problem with this article, and of so many articles like it: "When you’re picking a data store" "a", as in singular. There is no rule in building software that says you have to use 1 tool to do everything.
  • dbcfd 13y ago
    I think this is the first post on HN I wish I had a downvote button for, just for the reason you list. There is a reason there are different flavors of databases, and MongoDB most definitely would not be my choice for representing graph like relationships. It's also scary that it has 217 points because it bashes Mongo.
    • mcknz 13y ago
      I often upvote articles because I'm interested in the discussion. It does not always indicate agreement.
    • towelrod 13y ago
      I think you are missing the point of the article. If you read down to the Epilogue it explains how the "perfect" application still didn't work with MongoDB once the clients started asking for more features. My read was that even when you think you don't have "graph like relationships" in your data, you actually do. The original author did say this, but I would like to add: if you don't have "graph like relationships", then your data is pretty trivial and any data store will do.
      • dbcfd 13y ago
        From another comment I made, on why I don't think is a good article even using the proposed thesis of "mongo doesn't work for graph like relationships": Even though their data doesn't fit well in a document store, this article smacks so much of "we grabbed the hottest new database on hacker news and threw it at our problem", that any beneficial parts of the article get lost. The few things that stuck out at me: * "Some folks say graph databases are more natural, but I’m not going to cover those here, since graph databases are too niche to be put into production." - So you did absolutely no research * "What could possibly go wrong?" - the one line above the image saying those green boxes are the same gets lost. Give the image a caption, or better yet, use "Friends: User" to indicate type * "Constructing an activity stream now requires us to 1) retrieve the stream document, and then 2) retrieve all the user documents to fill in names and avatars." - Yep, and since users are indexed by their ids, this is extremely easy. * "What happens if that step 2 background job fails partway through?" - Write concerns. Or in addition to research, did you not read the mongo documents (write concern has been there at least since 2.2) Finally, why not post the schemas they used? They make it seem like there are joins all over the place, when I mainly see, look at some document, retrieve users that match an array. Pretty simple mongo stuff, and extremely fast since user ids are indexed. Even though graph databases are better suited for this data, without seeing their schemas, I can't really tell why it didn't work for them. I keep thinking "is it too hard to do sequential asynchronous operations in your code?".
• lampe3 13y ago
  I Stopped clicking on titles like this... Titles with NEVER/ALWAYS/something in that direction are just there to make you click and mostly are not good articles. Maybe this is i don't know... i don't clicked on it...
  • golfadas 13y ago
    It is a flaming title, but an actual good read if you want to know what kind of problems you might have when using Mongo.
• jwwest 13y ago
  Link bait title aside, I'm a little bored with these "We don't like x, so you shouldn't use it" articles. The main downside of MongoDB is that it's new. This means less knowledge of best practices, incomplete or missing support in third party integrations, and feature-lacking tools. It also takes a different approach to architecting systems than you would take when using a SQL approach.
  • dbcfd 13y ago
    There's also the fact that what they were trying to do in Mongo was not what you should do in Mongo. Use a relational or graph database for data that is best represented as a relation or graph. Nothing like using a hammer to paint a wall and then say you should never us a hammer.
• dasil003 13y ago
  There is a good reason that relational databases have long been the default data store for new apps: they are fundamentally a hedge on how you query your data. A fully-normalized database is a space-efficient representation of some data model which can be queried reasonably efficiently from any angle with a bit of careful indexing. Of course relational databases being a hedge, are not optimal for anything. For any particular data access pattern there's probably an easy way to make it faster using x or y nosql data store. However as the article points out, before you decide to go that route you better be pretty certain that you know exactly how you are going to use your data today and for all time. You also should probably have some pretty serious known scalability requirements otherwise it could be premature optimization. Neither of these things are true for a startup, so I'd say startups should definitely stay away from Mongo unless they really know what they are doing. Being ignorant of SQL and attracted by the "flexibility" of schema-less data stores powered by javascript is definitely the wrong reason to look at Mongo.
  • camus2 13y ago
    > Being ignorant of SQL and attracted by the "flexibility" of schema-less data stores powered by javascript is definitely the wrong reason to look at Mongo. It's usually the only reason. And 10gen were good at marketing it.
    • mattkrea 13y ago
      I think a lot of people (at least in the Node community) love the interfaces provided for it and specifically Mongoose which allows you to enforce a schema and thus relational data. I'll admit that I originally got into it because I didn't really know SQL and I'm still not very talented with it but for example.. Joins in Mongoose? Say there is a comment.. this comment has an author. If that author is type ObjectId when I run a query I can do this: model.comment.find({_id: <some id>}).populate('author').exec(e, result) { // author will be populated with that author's data instead of just the ID }
      • camus2 13y ago
        I use mongodb only when i have to write an app in Node. Not because it's the better solution , but because mongoose is the only library that deals with data that is mature on npm. The rest is beta at best and drivers for RDBMS are not mature enough.
• auggierose 13y ago
  This seems to be a case of incompetent programmers doing jobs which are way over their heads. Just saying. Well, if it doesn't kill them, they'll learn from it.
• ryanobjc 13y ago
  They say never jump in an argument late... but here goes... There is a lot of people arguing in the positive for polyglot persistence. The arguments sound pretty appealing - hammer, nail, etc - on the face of it. But as you dig deeper into reality, you start to realize that polyglot persistence isn't always practically a great idea. The primary issue to me boils down to safety. This is your data, often the only thing of value in a company (outside the staff/team). Losing it is cannot happen. Taking it from there, we stumble across the deeper issue that all databases are extremely difficult to run. DBAs dont get paid $250k/year for nothing. These systems are complex, have deep and far reaching impacts, and often take years to master. Given that perspective, I think it then makes the decision to use a single database technology for all primary storage needs totally practical and in fact the only rational choice possible.
• AsymetricCom 13y ago
  LMFAO at startups using hot new technology so they can throw back the technical debt back in their willfully ignorant investors' laps. Then the business suits come back thinking "yeah we want a feature that makes us as useful and powerful as IMDB, that site looks so dated, we'll steal their market easy. Yeah just add actors. Piece of cake right? Shouldn't take you more than a day." Hahahahhaha.
• dgregd 13y ago
  I hope that Mozilla guys will also learn that lesson. They helped Microsoft to kill WebSQL. So instead of SQLite, we have terrible IndexDB inside browsers.
• LouisSayers 13y ago
  How about this for an article: Why you should never use a Microwave. I tried toasting my toast in the microwave and it didn't work so good. How about you use a document store for documents, and a graph db for graphs? Of course mongo won't work for data with loads of relationships in it, because it's not meant for that. Next time take your bread out of the microwave and put it in the toaster - I guarantee you'll get better results.
• pkorzeniewski 13y ago
  All things aside, the main problem I had with MongoDB when I've tried to use it in one of my projects (in NodeJS) was the way to query data. I know, there are some libraries for that, but it's just pain in the ass to make complex queries - something which would be an easy JOIN in SQL, in MongoDB is a big pile of callbacks and long, chained method calls. It was just hard to maintain, and the data structure flexbility didn't make it easier which led to a completely messed up database.
• deleted 13y ago
  [deleted]
• eonil 13y ago
  (1) The author knew social data is a graph topology. (2) Also knew there's no true production level graph database solution in the market.(I doubt this but anyway, the author did) (3) And tried to store it into many ISOLATED trees - what MondgoDB offers. (4) Realized that's impossible. (5) Blame MongoDB for lack of graph connectivity feature. (6) Back to RDBMS for consistent graph connectivity. None of this procedures does make sense. I want to ask to the author. Did you really knew what the GRAPH is?
• beat 13y ago
  Consider Neo4J. The fail mode here for MongoDB (complex non-hierarchical relationships) is a win mode for Neo4J. Hierarchies are just a special case of graphs. Use a proper graph database, and you can actually represent the relationships based the domain model, without the clumsiness of hierarchical NoSQL or even worse clumsiness of RDBMS.
• Nodex 13y ago
  So let me get this straight. You don't know how to model data properly and it seems the guys who wrote the app you describe don't either so that automatically means that nobody should use MongoDB ever?. MongoDB DOES NOT CLAIM to be a silver bullet yet all you hipster bashers base your arguments that it cannot do XYZ job properly and this it must be bad. USE THE RIGHT TOOL FOR THE JOB - simple ;)
• memracom 13y ago
  Many boosters of NoSQL forget that relational database engines are not all the same. They evolve, and are no longer the same as they were 10 years ago when people started building NoSQL engines. In particular, PostgreSQL is not the same as Oracle or MySQL, and PostgreSQL has been evolving quite a lot in recent years to make it a stronger competitor wih NoSQL.
• csense 13y ago
  I was never fully sold on the advantages of jumping on the NoSQL bandwagon. This article echoes a lot of the concerns I had when people were first getting excited about MongoDB.
• quizotic 13y ago
  Funny thread! MongoDB's query performance is typically sub millisecond, and rarely as much as 5ms. I don't think you can through the postgres/mysql parser in 5ms, much less optimizer, planner, and execution stack. Couple that speed with a dead simple API, and you've got a thing of beauty. So yeah, if you don't care about ms latencies, if you've got a fixed rectangular schema, if want to write queries with lots of joins, if you need ACID guarantees ... etc ... then by all means pick your favorite RDBMS. OTOH, when you need to manipulate relatively self-contained objects quickly, when those objects don't have a fixed schema, when availability is more important to you than immediate consistency, then why would you choose anything other than MongoDB?
  • jacques_chester 13y ago
    Your argument here is in favour of premature optimisation. You are wagering performance now against the assumption that your schema will never change, that will never need joins, that there are no, and will never be, entities that appear in common in multiple points of the document graph ...
  • j-kidd 13y ago
    > I don't think you can through the postgres/mysql parser in 5ms, much less optimizer, planner, and execution stack. Yeah... except no. I just set `log_min_duration_statement` to 0, and can see that PostgreSQL typically takes less than 0.1 ms to parse a query. Quickly parsing a query to come up with an optimized plan is actually a great strength of PostgreSQL, when compared to other RDBMS. MSSQL, for example, has this complex query plan caching mechanism to compensate for its slow parsing. PostgreSQL doesn't need that. Also, with EXPLAIN ANALYZE, I can see that PostgreSQL typically takes less than 0.1 ms to do an index lookup as well. You seem to believe that MongoDB has some kind of magic that makes it the only database that can perform sub millisecond query. 10gen is doing a great job there.
• hobs 13y ago
  To be fair, I work in SQL constantly, but someone saying "7 table joins ugh" all I can think is really? Maybe I have spent too much time in the TSQL world, but I regularly see things with 20+ joins without blinking.
• henryw 13y ago
  I was about to go all in with mongodb (w/ node) on my next project, including for the user table. But after this, I'm going have to re-evaluate it. "Once we figured out that we had accidentally chosen a cache for our database, what did we do about it?" "The only thing it’s good at is storing arbitrary pieces of JSON. “Arbitrary,” in this context, means that you don’t care at all what’s inside that JSON. You don’t even look. There is no schema, not even an implicit schema" "I’ve heard many people talk about dropping MongoDB in to their web application as a replacement for MySQL or PostgreSQL. There are no circumstances under which that is a good idea." "I suggest taking a look at PostgreSQL’s hstore"
• ezmobius 13y ago
  You thought you wanted to use a document store until you realized that redis+postgres was actually what you needed :P
• devanti 13y ago
  someone misused mongodb, and now they're blaming mongodb for it. A social network will have a lot of relations -- so you should use a relational database. Other web apps may not need as many relations (most mobile apps), in which mongodb is a superior solution
  • argvzero 13y ago
    you keep using that word "relations". i do not think it means what you think it means. http://en.wikipedia.org/wiki/Relational_algebra http://en.wikipedia.org/wiki/Relational_algebra
• DigitalSea 13y ago
  So in other words, they misused MongoDB and because of that are telling people not to use it? Wow. Seems to be a case of "a bad mechanic will always blame his tools". In the right hands MongoDB can be a great asset. The problem here is that Diaspora chose MongoDB when it was very immature and it seems the choice was based on hype more so that mapped out requirements. This is where proper planning for a large scale application will spot these kinds of problems before they get to the development stage. Later versions of MongoDB are much better and the upcoming planned changes will take it many steps in the right direction towards being a viable alternative to a traditional RDBMS. Having said that, it's not a silver bullet and MongoDB is not for everything. 10Gen are exceptionally great at marketing Mongo and I get the feeling they have kind of trapped Foursquare who have been using it in production for a couple of years or so now. Having said that with exception of that one 11 hour outage Foursquare encountered, MongoDB seems to be working really well for them and they seem to be capturing more than just check-in data. I still think with proper planning pulling off a social network with MongoDB is possible. I am currently building a social networking type application, not on the scale of Facebook but it does share some parallels. I've planned and mapped out a viable structure and how it all connects, prototyping and testing seems to indicate that MongoDB is up to the task, but we'll see.
  • wmt 13y ago
    "So in other words, they misused MongoDB and because of that are telling people not to use it? Wow. ... The problem here is that Diaspora chose MongoDB when it was very immature" So the specific misuse of MongoDB was to use it? I guess that was also the point of the story.
    • DigitalSea 13y ago
      Yes, exactly. They chose the wrong tool for the job, that's not a fault with MongoDB. It's like eating a soup with a fork, you'll eventually eat the soup, but if you used a spoon in the first place there would never have been a problem. Is the company who made the soup to blame or are you to blame for not consuming it correctly? The problem with Diaspora was that it was a poorly executed good idea. Had they actually sat down and mapped out their requirements and chosen an appropriate database they would have realised that a traditional relationship was the right choice to make. Databases are hard, for a project as ambitious as Diaspora proper planning is key and evident by the issues that Diaspora had when it debuted (delete controller with no auth checking...) it's apparent they had no clue what they were doing not from just a database planning point-of-view but a code one as well. I would take whatever anyone who had anything to do with Diaspora had to say with a grain of salt. The real lesson here is to not get caught up in hype and trends. NoSQL isn't a magical solution that will make scaling problems disappear, traditional databases like MySQL have been battle-tested over a very long period of time. If MongoDB and <insert X NoSQL database here> were so great, the likes of Facebook and whatnot would use them on the scale Diaspora tried to use them on. People have to realise when Diaspora used MongoDB it was a very early version of the database. A lot has changed since Diaspora used it, it is suitable for a large-scale Facebook clone as the sole database? Definitely not, but using it for aspects like messaging and notifications I think would definitely be a good use case for it.
• hakcermani 13y ago
  I wonder why this is such a surprise and all this hoopla ... http://docs.mongodb.org/manual/faq/fundamentals/#what-are-typical-uses-for-mongodb http://docs.mongodb.org/manual/faq/fundamentals/#what-are-ty... The big advantage of mongo (as other nosql dbs) is the dynamic schema. Try adding a new column to your sql database ..
• mcgwiz 13y ago
  I give Diaspora and the author kudos for the effective hatebait title, though I would have preferred "Study the documentation/source of all mission critical components carefully." Reading through the article, I saw similarities between their issues and ones I've encountered. However, having skeptically studied MongoDB's docs and tested its behavior, I had a very good idea early that MongoDB preferred idempotent, append-only, and eventual-consistent data, with two-phase commit being the nuclear option (and that boxen-wise, the high-memory ones are preferred). Regarding denormalization storage, invalidation is indeed a tricky issue. A CQRS approach with a durable event queue between the command/domain and an idempotent denormalization system elegantly yields an exceptional scalability:complexity ratio. In the end, I built a performant, flexible app that's been a joy to work on and operate. I see pain similar to Diaspora's as a result of "move fast, break things" culture taken to the extreme. No planning or basic research of any kind is expected anymore. I know (from experience watching people pick technologies they barely understand and try to IntelliSense their way to delivery) that I'm in the minority, but I've always preferred to read a book cover-to-cover (or traverse a website depth-first) before adopting a technology in earnest, because I know I don't know what I don't know.
• dreamdu5t 13y ago
  Like mongo queries but use SQL with node.js? Check out https://github.com/goodybag/mongo-sql https://github.com/goodybag/mongo-sql
• lynchdt 13y ago
  Over the past year we have from scratch built a significant web-application with MongoDB. We also have a social graph. We build infinite-scroll activity feeds on the fly. We handle multiple writes per second and 10s of reads per second. We have hundreds of thousands of users. We have 100 million documents, growing nicely. We've forced square pegs into round holes on occasion, but nothing we were too surprised about. It seems like you walked into to your database technology choice with your eyes fully shut. Given even a modicum of of preparation - e.g. reading the MongoDB documentation - you would have asserted the social graph use case to be a challenging one with a data-store in which relations are unnatural. Then - because you realized you may have a sub-optimal solution, you optimize it by changing technology. And then decide to join this ridiculous anti-MongoDB internet bandwagon. For somebody who builds "4-6 web-applications per year" and has deployed "most of the data-stores you've heard about - and some you haven't" this seems surprising. Or perhaps not, actually.
• khailey 13y ago
  Putting together a social network schema and keeping performance in check is less than evident. It's a problem that many have to tackle such as facebook, twitter, linkedin, flickr etc. Here is a blog article discussing some of the issues and approaches: http://www.kylehailey.com/facebook-schema-and-peformance/ http://www.kylehailey.com/facebook-schema-and-peformance/
• leccine 13y ago
  I find it fascinating that people misunderstand data so much even 2013 that they end up implementing a totally inefficient solution without actual data modeling and design. The most missed point is usually querying. Focusing on the how do I write data first can lead to disasters like this. Why don't everybody start with the "how do I query this dataset in this shape?" question first?
• snambi 13y ago
  Absolutely Agree.
• bibstha 13y ago
  Is it also the case of all other nosql databases like Cassandra or CouchDB etc?