12 ms·

Why Did So Many Startups Choose MongoDB?

102 points by brakmic 9y ago

xmatos 9y ago
Hype and lack of critical thinking abolutely had their role, but mostly because it's so much easier than a relational db. Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts. The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server. Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping. Who cares about data integrity or technical debt?
- nevi-me 9y ago
  It has most of that. It's not that RDBMS are hard and schemaless convolutions are easier. Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection. I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
jasondc 9y ago
This post makes the assumption that MongoDB isn't doing well based on Hacker News. MongoDB the company, and product, are doing extremely well (outside of anecdotal Hacker News posts), based on objective metrics like downloads, skills, jobs, and company revenue (and the company is nearing 900 employees on linkedin). To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
- __jal 9y ago
  Can't speak to current popularity, but I think the overall critique is spot on. It became really popular riding a combination of up-front friendliness and the No-SQL hype bubble. The next morning, a lot of folks started feeling the hangover, whether from actual problems with Mongo or a realization that they did, in fact, need a relational DB. To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype. It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
- owebmaster 9y ago
  MongoDB is like the PHP of databases. I started programming PHP and currently I bash it every time I can, because it is a terrible language. But it is easy to start and do cool stuff, just like mongodb. Initial traction is more important than initial architecture, I guess.
  - mattacular 9y ago
    Why is PHP a terrible language? It powers a vast part of the internet including sites huge, tiny, and everything in between. It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
    - threepipeproblm 9y ago
      This lack of insight is why PHP developers have lower average salaries than non-PHP developers. If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP. Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-frameworks-dont.html http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....
Daishiman 9y ago
The misplaced hype cycle is real. 10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did) Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric. Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding. This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.
- epmaybe 9y ago
  Why wouldn't you always use something that is tried-and-true? MongoDB is relatively new compared to something like postgresql or even mysql.
  - wmf 9y ago
    If you think that every tried-and-true database is slower to develop against.
  - klodolph 9y ago
    The hope is that new technologies would provide some advantage, like prototyping speed. That was certainly true of Ruby on Rails, for example, back when it was the new kid on the block and the "tried and true" frameworks were stuff like Struts or Spring. Yes, that's not a database technology example.
  - squeaky-clean 9y ago
    If you actually need certain features like a document store, then Postgres is less mature than MongoDB in that regard.
    - 9y ago
have_faith 9y ago
I'm not a back-end dev strictly. I played with mongo on some side projects back when it was all hyped up. One thing that appealed to me was the on-boarding process for a non "database guy". I got lots of small side projects up and running very quickly and the concepts made sense. Similar to jQuery being easy to understand and popular to newcomers but you later learn it isn't always the best tool for the job.
stevecalifornia 9y ago
Knowledge does not equal wisdom. Perhaps it was young folks with lots of knowledge didn't see the wisdom in using battle-hardened, boring technology.
- sidlls 9y ago
  It isn't an age thing. There are mid-life engineers where I work who are constantly trying to force the latest fad in tech into our engineering organization.
cbsmith 9y ago
Come on, we all know why: https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36COcfSrQfNkeocctpfmG1glwT https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36C...
- reallydattrue 9y ago
  That made me lol! This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex. No wonder the whole javascript stack is a complete and utter mess!
- deleted 9y ago
  [deleted]
jszymborski 9y ago
What NoSQL databases are people using today when they need something ACID compliant? CouchDB?
- opendomain 9y ago
  NoSQL originally meany to not use SQL as the query language However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
  - dragonwriter 9y ago
    > NoSQL originally meany to not use SQL as the query language I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
  - hodgesrm 9y ago
    ACID and relational are orthogonal. You can have ACID properties without a relational model. Much of the early literature on the subject merely discusses databases in general rather than any particular type. Here's an example from Jim Grey: http://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf http://jimgray.azurewebsites.net/papers/thetransactionconcep...
jondubois 9y ago
I still think MongoDB is a good database. Conceptually, It's not as good as RethinkDB but it's not too far off. It's good to not have to deal with SQL and horrible ORMs anymore. TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
- insulanian 9y ago
  > It's good to not have to deal with SQL... I will never understand this attitude.
- always_good 9y ago
  I don't understand what you're saying MongoDB solved. Mongo has ORMs too like Mongoose. Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
  - jondubois 9y ago
    Mongoose is a lot simpler and less brittle than other ORMs in SQL land. That said, for most use cases, you don't need to use Mongoose. When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions. With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
siculars 9y ago
Mongo succeeded for three reasons: 1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks. 2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails. 3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials. These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
- hodgesrm 9y ago
  Not so different in some ways from the rise of MySQL itself. In point #2 substitute PHP for Node and RoR. In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host. The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
joeblau 9y ago
Back in 2012 I worked at a startup and we had a few backend data stores (Vertica, Postgres, Redis, Big Table, MySQL, Cassandra, ElasticSearch, Hadoop). When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization). At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems. That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
- cphoover 9y ago
  "which was actually funny to watch one of our engineers try to unlearn table --denormalization-- normalization*" Going from sql to nosql would be denormalizing data not normalizing it so, the engineer would be unlearning table normalization.
  - joeblau 9y ago
    Ah yes, you're correct!
- jsherard 9y ago
  You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft. http://bit.ly/2uP3Obi http://bit.ly/2uP3Obi
diziet 9y ago
Any criticism of MongoDB that does not acknowledge WiredTiger and the plethora of positive changes that made MongoDB much, much better in the 3.0, 3.2 and 3.4 releases is criticism that is not fully thought out. It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
- nevi-me 9y ago
  In the end it doesn't really matter. I feel the article is an educational one, but as it's the chance for us to take our sides of the fence in praising and criticising, we'll still do it anyway. I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days. A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
  - metheus 9y ago
    > I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days. You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
- lod723 9y ago
  You realize that this sounds like more of the same hype to anyone who was burned by the overpromising in the 1.x and 2.x days? Whether your statement about 3.x is true or not, the MongoDB people dug their own hole that they need to try extra hard to get out of, and so far it's not enough.
- deleted 9y ago
  [deleted]
alberth 9y ago
Not to sound flippant but the reason why so many startups choose mongodb is because the problems they are working on are better suited to be addressed with a document-oriented database than relational database. And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
- Daishiman 9y ago
  What are those problems exactly?
mbesto 9y ago
I've assessed a fair number of companies who use Mongo in some way or another and the I always ask the rationale for the decision. Surprise surprise, it's usually "we chose it because it was easy to start with". My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
- takeda 9y ago
  In case of Edmunds, the rationale was that their contractors previously used Mongo and were more familiar with, and upcoming 2.6 version will have some "new cool features". Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1]. [1] https://en.wikipedia.org/wiki/Oracle_Coherence https://en.wikipedia.org/wiki/Oracle_Coherence
takeda 9y ago
Looking at the table from 10th Gen, I believe it is misleading, at least in relation to Edmunds. I worked there, and Oracle (which was the source of truth) was replaced by PostgreSQL. Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost. I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went. I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives. [1] https://en.wikipedia.org/wiki/Oracle_Coherence https://en.wikipedia.org/wiki/Oracle_Coherence
dizzystar 9y ago
I think the short answer is the same today as it was back in those days. A person once put it to me simply: "programmers are afraid of databases." It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper. Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
- bdcravens 9y ago
  > programmers are afraid of databases. Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization. > Add the idea that there is no need for a DBA With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well. > encourage developers to see Mongo as a magic data storage and nothing more In modern frameworks, this tends to be true no matter what the data store is.
nevi-me 9y ago
Interesting article, and I look forward to the other 2 parts. This was well-researched and balanced. MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed. Someone else commented about the HN bias, which we can't deny exists. I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today. Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ... Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained. I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
Osmium 9y ago
I've seen a few comments mention RethinkDB ... what are the advantages over MongoDB? I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.
- kronos29296 9y ago
  The company got bankrupted and they set up an org to fund its development. (Sagemath moved to Postgresql from RethinkDB after the company got sold. The guy behind sage said it was better in his blog about the switch)
  - williamstein 9y ago
    Thanks; here’s my blog post: http://blog.sagemath.com/2017/02/09/rethinkdb-vs-postgres.html http://blog.sagemath.com/2017/02/09/rethinkdb-vs-postgres.ht... Incidentally, http://www.lmfdb.org http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
- williamstein 9y ago
  The RethinkDB company went out of business last year, and unfortunately development has basically stopped, as you can see from the commits to their repo: https://github.com/rethinkdb/rethinkdb/graphs/contributors https://github.com/rethinkdb/rethinkdb/graphs/contributors The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb https://groups.google.com/forum/m/#!forum/rethinkdb RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today. I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
JustSomeNobody 9y ago
There were so many blog posts about how MongoDB was the future and SQL/relational dbs were the past. I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
wyc 9y ago
MongoDB is an interesting database for sure. It seems that they made a deliberate product decision to optimize developer experience and ease of use over things like consistency and database joins. Frankly, I think it's working pretty well for their business. I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services. People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
niahmiah 9y ago
to avoid schema migrations
- takeda 9y ago
  And in turn put the schema (actually multiple versions of it) in your application making it more complex. Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay. JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux. [1] https://en.wikipedia.org/wiki/XML_database https://en.wikipedia.org/wiki/XML_database [2] https://en.wikipedia.org/wiki/Database_model#Object-oriented_database_models https://en.wikipedia.org/wiki/Database_model#Object-oriented...
  - niahmiah 9y ago
    I'd rather keep the logic in my app than have planned downtime for schematic migrations
ransom1538 9y ago
I have been paid by two companies to remove Mongo. One that was hosted at the Mongo HQ. (The mongo HQ guys are really nice). At small startups, things are on fire and people need answers quick. "How many users tried this, how many times was feature x used," on and on. If it is in mysql, well i just setup a vpn and a read mysql access, done -- they can query away. If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
joshribakoff 9y ago
I use Mongo & its been working great. Our data is not relational (imagine you were tasked with building "Powerpoint" on the web). Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over. With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works". Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
- lotyrin 9y ago
  Of course I don't know your product, but when I imagine Powerpoint-as-a service, do users not relate to decks which relate to slides which relate to the content on those slides? Content itself of course would probably be a pretty flexible tree structure. An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me. And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast. If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product. HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen. As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
deleted 9y ago
[deleted]
smileysteve 9y ago
Similar questions can be asked > Why do so many developers use map/reduce instead of the database for select, pluck, math? > Why do so many developers use length instead of counting sql? The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.
lcfcjs 9y ago
Because MongoDB stores documents in JSON form. It's the easiest data structure, easiest to transform in whatever language you're using, it's fast, and its scalable.
manishsharan 9y ago
I think Enterprise folks do not appreciate the flexibility afforded by NoSQL . For my startup I started off on PostGres SQL with Hibernate and it worked great ... until we first met our paying customers. They needed data elements that I had not envisioned and so we ended up doing Schema updates.. again and again . Since we used ORM (hibernate), that also meant new hibernate code generation . After a while this became very tedious. I have a background in enterprise s/w development; we always lock down on most of our requirements before we start writing code. RDBMS is fantastic in this scenarios. In my startup, we build something light and show it to bunch of prospects and iterate. MongoDB fits this mode of s/w development for us. Sure there are gotchas and other issues but they have not affected us yet.
- sidlls 9y ago
  Database migrations are a thing. Many languages' database libraries have built-in or bolt-on schema migration tools for this very purpose.
  - enjoiful 9y ago
    Django's ORM and migration tools are amazing.
- tcoff91 9y ago
  You can do NoSQL with Postgres...
- Daishiman 9y ago
  The gotcha is going to come when your schema has dozens of fields whose existence is uncertain and they begin generating runtime errors when your code expects them to exist.
exabrial 9y ago
Really good marketing, mongo's pitch was "use this if you want to be Innovative"
Tokkemon 9y ago
Because it's not MySql. "We can't use our dad's database!"
dyeje 9y ago
Reminds me of the keynote DHH gave at RailsConf this year. Despite the stories that we like to tell ourselves about picking the right tool for the job, we're just as prone to chasing the shiny new thing as all the other humans.
- sidlls 9y ago
  Who's "we"? I have yet to see a high quality technical argument for using a number of technologies that have come along the last 5-8 years or so in most cases. Too often "scalability" and "modern stack" are used as an excuse to tinker with the new shiny far out of bounds of their proper use. I have a reputation as being the opposition any time adoption of something new comes up. It's rarely necessary.
virmundi 9y ago
As a person who just recently turned down an offer to write an ArangoDB book, Mongo is not worth it. I think most needs are met between Arango and PostgreSQL. Arango provides a scalable NoSQL option over PostgreSQL (let's see what version 10 holds). PostgreSQL provides a solid, efficient SQL solution. Mongo provides no benefits of transactions or solid design. If you want either, look to Arango or PostgreSQL.
nir 9y ago
Marketing aside, MongoDB is just really easy to pick up and use. In a small project, its weaknesses don't really show up (at least in my limited experience), if it gets big you often end up rebuilding anyway.
chx 9y ago
For us, the reason to run with MongoDB in 2009 were twofold: a) for one of our products which was consuming the Twitter gardenhose, unverified writes were a blessing. We got away with a single dedicated server running mongodb (and it was not a particularly expensive server). I would need to do serious research again whether there's a better solution for this today. It was using Stanford NLP to highlight breaking news before anyone realized. We never got a single subscriber because every newsdesk that saw it wanted to acquire us immediately LOL. Eventually one did. b) For another, the ability to store and index complex documents which in SQL would've needed JOINs and you couldn't index across JOINs at least in MySQL and our previous experience in PostgreSQL vs MySQL support didn't favor PostgreSQL at the time at all. It worked -- although it didn't have a journal back then. We were ready for a failure but the failure never came. The landscape today is much different with PostgreSQL and MySQL and SQLite all embracing JSON storage and indexing on expressions are there for all three.
andred14 9y ago
Anyone with proper unbiased DB knowledge knows without trying or reading anything that it is hype. DBs are all about trade-offs you can't have it all
bastijn 9y ago
Obligatory read when talking MongoDB https://aphyr.com/posts/322-jepsen-mongodb-stale-reads https://aphyr.com/posts/322-jepsen-mongodb-stale-reads and follow-up https://jepsen.io/analyses/mongodb-3-4-0-rc3 https://jepsen.io/analyses/mongodb-3-4-0-rc3. Ensured me to never use this for production scenarios. No matter how 'hyped' or 'convenient' it is. Yaiks.
andred14 9y ago
Anyone with average to more DB experience knows there is no one-stop shop. DBs are about trade-offs and we need to choose the right ones for our application(s)
namlem 9y ago
Because they have great docs. At least that's why I chose them for my (now defunct) startup back in 2013. I wasn't familiar with databases and their docs were extremely approachable and easy to understand and navigate, with lots of useful examples.
nemild 9y ago
I wrote this series over several months and am happy to answer questions. Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr http://eepurl.com/cxs5Zr As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.
ahallock 9y ago
Marketing & ease of use (for example, if you try to insert into a collection that does not exist, it's automatically created for you).
jsherard 9y ago
I find it difficult to seriously address comments to this thread. People still referencing 10gen and talking about data points from 5-10 years ago (2009, 2012, 2013). You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft. http://bit.ly/2uP3Obi http://bit.ly/2uP3Obi MongoDB has many built in features that are extremely attractive for building modern applications. Built in HA & DR, horizontal scalability, multiple storage engines, rich indexing and query capabilities, enhanced BI features in aggregation framework, and many more...
logwriter 9y ago
You guys are discussing MongoDB from 2009. The database has evolved since then. - There is a new replication protocol for replica sets, improving the election process. - WiredTiger has been introduced and became the default storage engine. - Journaling guarantees durability. - Sharding architecture has changed. Config Servers are now deployed as Replica Sets and some of the tasks that used to be managed by mongos now are managed by the config servers. - Storage efficiency has been added through compression. - Security has been improved with encryption at rest. - The aggregation framework has more options for BI. - There are a variety of different indexes that can be created to improve query performance. - and finally, stop saying MongoDB is schemaless because it is not. There is a schema in place and this schema is flexible enough to be polyform. These are features available today in 3.4. You might have issues in the past, but if you really understand the newest architecture, you are capable of seeing how much MongoDB has grown since 2.4.
- Daishiman 9y ago
  But at this point that's all pretty much moot because competitors have caught up and there's hardly an advantage over Postgres' JSONB fields, which builds up on top of all the features Mongo will likely never have (transactions I'm looking at you).
felipeccastro 9y ago
It's not just because of hype that teams choose Mongo. The flexibility of a schemaless db is pretty helpful in a startup scenario, where the requirements drastically change all the time. Yes, migrations work for RDBMS too, but with Mongo you only write migrations for data changes, not schema changes - which means a lot less migrations. The option of having documents with embedded objects/lists makes it a pretty natural mapping to Domain Driven Design concepts, like aggregate roots + aggregates. One might even argue that while following DDD, it's preferable to handle most rules at the application layer and use the database simply as a performant, low-maintenance storage service. I know some folks here might view this approach as a sin, but it works pretty well too, especially for smaller teams without a dedicated DBA. Maybe it's just me that haven't ran large enough systems to find the kind of issues with mongo that people always rant about, idk. Honest question: considering all these advantages above (from a developer's point of view), are there good reasons for choosing something like PostgreSQL over Mongo for typical business apps?
- busterarm 9y ago
  I'm not entirely convinced that MongoDB is better suited or a natural mapping to DDD concepts in practice, it's just more convenient for developers who want to build things fast. My thoughts on this pattern are to still use a robust, reliable db (Postgres) behind a message queue for the event store. Postgres does almost all of the things I need with the reliability that I need while letting me sleep at night. An event store absolutely must be the most reliable part of the infrastructure, IMO.
  - felipeccastro 9y ago
    "just more convenient for developers who want to build things fast" - that's a compelling advantage for Mongo, isn't it? I agree a robust, reliable db is super important - but isn't Mongo too, at this point? Even if it wasn't back in 2012? I thought most of the reliability issues and bad defaults were fixed by now...
    - busterarm 9y ago
nasalgoat 9y ago
We chose it because the lead developer liked it, and when I did my research I didn't find any glaring issues because it was too new. The lead dev had never actually used it, but liked the idea of not having to come up with a schema and he really liked JSON as he was very familiar with it. By the time we were done, we were easily one of the top 5 users of MongoDB at scale and it was responsible for 9 out of 10 of our outages due to various bugs or optimization issues.
ivm 9y ago
Our team had little experience with backends in 2013 and a fear that it will be hard to scale game servers in case of success (players hate lags and downtimes). So we followed the hype of Node.js and MongoDB because everybody was saying how it's easy to scale them compared to the established tech.
rfehrmann 9y ago
We are also one of those companies that bought into the promise (or hype) in 2011 and gave MongoDB a spin. After some very successful tests, I had long conversations with my CTO on why we should deploy MongoDB in prod and make it our primary OLTP store. Yes, MongoDB had it's challenges in 1.6 and it required some "faith". But it's 2017 now and MongoDB totally delivered on it’s vision, at least for us. Since 2012, that's when we upgraded to 2.0, we had 10 min of total downtime. That's 99.999% uptime in 5 years. Year after year, 99.999% uptime. And that 10 min is the total for scheduled (btw, we don't have scheduled downtime for our MongoDB clusters since there is no need for that) and un-scheduled downtime. We never take the service down. We perform all maintenance operations during regular business hours with no impact to end-users at all. So much about reliability! MongoDB as a product has bugs. What a surprise, what software product doesn't. But it's about you community (we are running the community edition) supports the product. We recently discovered an issue as well causing the primary to crash. As designed, another member in the repl set took over and there was no downtime at all. Even though we don't have a support contract, the issue was acknowledged by MongoDB in hours and fixed in a couple of days. I wish we had that type of support for products where we do have a support contract! And last but not least, if you can't figure out how to successfully use MongoDB, for god sake, take some free training classes on MongoDB University. MongoDB might not be a good fit for every use-case, but it's already an awesome platform for an incredible number of applications. Polyglot environments is the keyword here. SQL has it’s advantages and disadvantages and so does NoSQL. Use the best tool for the problem. One size fits all doesn't work anymore and MongoDB is clearly the best and most popular document store on the market. And even better, other platforms like postgres have acknowledged that the idea of storing JSON has its merits and have implemented its own version of that idea. However, IMO, if your use-case is a good fit for a document store, use MongoDB. We have done so and couldn't be happier with that decision.
jaraco 9y ago
I'd wanted to lift the burden of writing ORM layers and dealing with relational algebra and focus on persisting my intrinsic application models. I'd studied XML and other semi-structured web data at the graduate level, but was frustrated there wasn't a technology that was meeting that need. Companies like Microsoft and Oracle were adding XML functionality to their databases, but because it was bolted on an already constrained structure, it failed to address the mangling that relational structures impose. I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows. There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch? Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings. When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful. Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later. But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB. I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box. And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition. I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.

102 points by brakmic 9y ago

Hype and lack of critical thinking abolutely had their role, but mostly because it's so much easier than a relational db. Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts. The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server. Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping. Who cares about data integrity or technical debt?

nevi-me 9y ago
It has most of that. It's not that RDBMS are hard and schemaless convolutions are easier. Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection. I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.

jasondc 9y ago

This post makes the assumption that MongoDB isn't doing well based on Hacker News. MongoDB the company, and product, are doing extremely well (outside of anecdotal Hacker News posts), based on objective metrics like downloads, skills, jobs, and company revenue (and the company is nearing 900 employees on linkedin). To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.

__jal 9y ago
Can't speak to current popularity, but I think the overall critique is spot on. It became really popular riding a combination of up-front friendliness and the No-SQL hype bubble. The next morning, a lot of folks started feeling the hangover, whether from actual problems with Mongo or a realization that they did, in fact, need a relational DB. To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype. It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
owebmaster 9y ago
MongoDB is like the PHP of databases. I started programming PHP and currently I bash it every time I can, because it is a terrible language. But it is easy to start and do cool stuff, just like mongodb. Initial traction is more important than initial architecture, I guess.
- mattacular 9y ago
  Why is PHP a terrible language? It powers a vast part of the internet including sites huge, tiny, and everything in between. It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
  - threepipeproblm 9y ago
    This lack of insight is why PHP developers have lower average salaries than non-PHP developers. If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP. Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-frameworks-dont.html http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....

Daishiman 9y ago

The misplaced hype cycle is real. 10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did) Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric. Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding. This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.

epmaybe 9y ago
Why wouldn't you always use something that is tried-and-true? MongoDB is relatively new compared to something like postgresql or even mysql.
- wmf 9y ago
  If you think that every tried-and-true database is slower to develop against.
- klodolph 9y ago
  The hope is that new technologies would provide some advantage, like prototyping speed. That was certainly true of Ruby on Rails, for example, back when it was the new kid on the block and the "tried and true" frameworks were stuff like Struts or Spring. Yes, that's not a database technology example.
- squeaky-clean 9y ago
  If you actually need certain features like a document store, then Postgres is less mature than MongoDB in that regard.
  - 9y ago

have_faith 9y ago

I'm not a back-end dev strictly. I played with mongo on some side projects back when it was all hyped up. One thing that appealed to me was the on-boarding process for a non "database guy". I got lots of small side projects up and running very quickly and the concepts made sense. Similar to jQuery being easy to understand and popular to newcomers but you later learn it isn't always the best tool for the job.

stevecalifornia 9y ago

Knowledge does not equal wisdom. Perhaps it was young folks with lots of knowledge didn't see the wisdom in using battle-hardened, boring technology.

sidlls 9y ago
It isn't an age thing. There are mid-life engineers where I work who are constantly trying to force the latest fad in tech into our engineering organization.

cbsmith 9y ago

Come on, we all know why: https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36COcfSrQfNkeocctpfmG1glwT https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36C...

reallydattrue 9y ago
That made me lol! This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex. No wonder the whole javascript stack is a complete and utter mess!
deleted 9y ago
[deleted]

jszymborski 9y ago

What NoSQL databases are people using today when they need something ACID compliant? CouchDB?

opendomain 9y ago
NoSQL originally meany to not use SQL as the query language However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
- dragonwriter 9y ago
  > NoSQL originally meany to not use SQL as the query language I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
- hodgesrm 9y ago
  ACID and relational are orthogonal. You can have ACID properties without a relational model. Much of the early literature on the subject merely discusses databases in general rather than any particular type. Here's an example from Jim Grey: http://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf http://jimgray.azurewebsites.net/papers/thetransactionconcep...

jondubois 9y ago

I still think MongoDB is a good database. Conceptually, It's not as good as RethinkDB but it's not too far off. It's good to not have to deal with SQL and horrible ORMs anymore. TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.

insulanian 9y ago
> It's good to not have to deal with SQL... I will never understand this attitude.
always_good 9y ago
I don't understand what you're saying MongoDB solved. Mongo has ORMs too like Mongoose. Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
- jondubois 9y ago
  Mongoose is a lot simpler and less brittle than other ORMs in SQL land. That said, for most use cases, you don't need to use Mongoose. When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions. With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.

siculars 9y ago

Mongo succeeded for three reasons: 1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks. 2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails. 3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials. These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.

hodgesrm 9y ago
Not so different in some ways from the rise of MySQL itself. In point #2 substitute PHP for Node and RoR. In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host. The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.

joeblau 9y ago

Back in 2012 I worked at a startup and we had a few backend data stores (Vertica, Postgres, Redis, Big Table, MySQL, Cassandra, ElasticSearch, Hadoop). When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization). At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems. That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.

cphoover 9y ago
"which was actually funny to watch one of our engineers try to unlearn table --denormalization-- normalization*" Going from sql to nosql would be denormalizing data not normalizing it so, the engineer would be unlearning table normalization.
- joeblau 9y ago
  Ah yes, you're correct!
jsherard 9y ago
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft. http://bit.ly/2uP3Obi http://bit.ly/2uP3Obi

diziet 9y ago

Any criticism of MongoDB that does not acknowledge WiredTiger and the plethora of positive changes that made MongoDB much, much better in the 3.0, 3.2 and 3.4 releases is criticism that is not fully thought out. It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.

nevi-me 9y ago
In the end it doesn't really matter. I feel the article is an educational one, but as it's the chance for us to take our sides of the fence in praising and criticising, we'll still do it anyway. I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days. A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
- metheus 9y ago
  > I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days. You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
lod723 9y ago
You realize that this sounds like more of the same hype to anyone who was burned by the overpromising in the 1.x and 2.x days? Whether your statement about 3.x is true or not, the MongoDB people dug their own hole that they need to try extra hard to get out of, and so far it's not enough.
deleted 9y ago
[deleted]

alberth 9y ago

Not to sound flippant but the reason why so many startups choose mongodb is because the problems they are working on are better suited to be addressed with a document-oriented database than relational database. And at the time, most of the mindshare is/was with mongodb for document-oriented technology.

Daishiman 9y ago
What are those problems exactly?

mbesto 9y ago

I've assessed a fair number of companies who use Mongo in some way or another and the I always ask the rationale for the decision. Surprise surprise, it's usually "we chose it because it was easy to start with". My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.

takeda 9y ago
In case of Edmunds, the rationale was that their contractors previously used Mongo and were more familiar with, and upcoming 2.6 version will have some "new cool features". Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1]. [1] https://en.wikipedia.org/wiki/Oracle_Coherence https://en.wikipedia.org/wiki/Oracle_Coherence

takeda 9y ago

Looking at the table from 10th Gen, I believe it is misleading, at least in relation to Edmunds. I worked there, and Oracle (which was the source of truth) was replaced by PostgreSQL. Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost. I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went. I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives. [1] https://en.wikipedia.org/wiki/Oracle_Coherence https://en.wikipedia.org/wiki/Oracle_Coherence

dizzystar 9y ago

I think the short answer is the same today as it was back in those days. A person once put it to me simply: "programmers are afraid of databases." It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper. Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.

bdcravens 9y ago
> programmers are afraid of databases. Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization. > Add the idea that there is no need for a DBA With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well. > encourage developers to see Mongo as a magic data storage and nothing more In modern frameworks, this tends to be true no matter what the data store is.

nevi-me 9y ago

Interesting article, and I look forward to the other 2 parts. This was well-researched and balanced. MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed. Someone else commented about the HN bias, which we can't deny exists. I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today. Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ... Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained. I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.

Osmium 9y ago

I've seen a few comments mention RethinkDB ... what are the advantages over MongoDB? I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.

kronos29296 9y ago
The company got bankrupted and they set up an org to fund its development. (Sagemath moved to Postgresql from RethinkDB after the company got sold. The guy behind sage said it was better in his blog about the switch)
- williamstein 9y ago
  Thanks; here’s my blog post: http://blog.sagemath.com/2017/02/09/rethinkdb-vs-postgres.html http://blog.sagemath.com/2017/02/09/rethinkdb-vs-postgres.ht... Incidentally, http://www.lmfdb.org http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
williamstein 9y ago
The RethinkDB company went out of business last year, and unfortunately development has basically stopped, as you can see from the commits to their repo: https://github.com/rethinkdb/rethinkdb/graphs/contributors https://github.com/rethinkdb/rethinkdb/graphs/contributors The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb https://groups.google.com/forum/m/#!forum/rethinkdb RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today. I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.

JustSomeNobody 9y ago

There were so many blog posts about how MongoDB was the future and SQL/relational dbs were the past. I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."

wyc 9y ago

MongoDB is an interesting database for sure. It seems that they made a deliberate product decision to optimize developer experience and ease of use over things like consistency and database joins. Frankly, I think it's working pretty well for their business. I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services. People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.

niahmiah 9y ago

to avoid schema migrations

takeda 9y ago
And in turn put the schema (actually multiple versions of it) in your application making it more complex. Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay. JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux. [1] https://en.wikipedia.org/wiki/XML_database https://en.wikipedia.org/wiki/XML_database [2] https://en.wikipedia.org/wiki/Database_model#Object-oriented_database_models https://en.wikipedia.org/wiki/Database_model#Object-oriented...
- niahmiah 9y ago
  I'd rather keep the logic in my app than have planned downtime for schematic migrations

ransom1538 9y ago

I have been paid by two companies to remove Mongo. One that was hosted at the Mongo HQ. (The mongo HQ guys are really nice). At small startups, things are on fire and people need answers quick. "How many users tried this, how many times was feature x used," on and on. If it is in mysql, well i just setup a vpn and a read mysql access, done -- they can query away. If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "

joshribakoff 9y ago

I use Mongo & its been working great. Our data is not relational (imagine you were tasked with building "Powerpoint" on the web). Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over. With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works". Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).

lotyrin 9y ago
Of course I don't know your product, but when I imagine Powerpoint-as-a service, do users not relate to decks which relate to slides which relate to the content on those slides? Content itself of course would probably be a pretty flexible tree structure. An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me. And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast. If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product. HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen. As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.

deleted 9y ago

[deleted]

smileysteve 9y ago

Similar questions can be asked > Why do so many developers use map/reduce instead of the database for select, pluck, math? > Why do so many developers use length instead of counting sql? The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.

lcfcjs 9y ago

Because MongoDB stores documents in JSON form. It's the easiest data structure, easiest to transform in whatever language you're using, it's fast, and its scalable.

manishsharan 9y ago

I think Enterprise folks do not appreciate the flexibility afforded by NoSQL . For my startup I started off on PostGres SQL with Hibernate and it worked great ... until we first met our paying customers. They needed data elements that I had not envisioned and so we ended up doing Schema updates.. again and again . Since we used ORM (hibernate), that also meant new hibernate code generation . After a while this became very tedious. I have a background in enterprise s/w development; we always lock down on most of our requirements before we start writing code. RDBMS is fantastic in this scenarios. In my startup, we build something light and show it to bunch of prospects and iterate. MongoDB fits this mode of s/w development for us. Sure there are gotchas and other issues but they have not affected us yet.

sidlls 9y ago
Database migrations are a thing. Many languages' database libraries have built-in or bolt-on schema migration tools for this very purpose.
- enjoiful 9y ago
  Django's ORM and migration tools are amazing.
tcoff91 9y ago
You can do NoSQL with Postgres...
Daishiman 9y ago
The gotcha is going to come when your schema has dozens of fields whose existence is uncertain and they begin generating runtime errors when your code expects them to exist.

exabrial 9y ago

Really good marketing, mongo's pitch was "use this if you want to be Innovative"

Tokkemon 9y ago

Because it's not MySql. "We can't use our dad's database!"

dyeje 9y ago

Reminds me of the keynote DHH gave at RailsConf this year. Despite the stories that we like to tell ourselves about picking the right tool for the job, we're just as prone to chasing the shiny new thing as all the other humans.

sidlls 9y ago
Who's "we"? I have yet to see a high quality technical argument for using a number of technologies that have come along the last 5-8 years or so in most cases. Too often "scalability" and "modern stack" are used as an excuse to tinker with the new shiny far out of bounds of their proper use. I have a reputation as being the opposition any time adoption of something new comes up. It's rarely necessary.

virmundi 9y ago

As a person who just recently turned down an offer to write an ArangoDB book, Mongo is not worth it. I think most needs are met between Arango and PostgreSQL. Arango provides a scalable NoSQL option over PostgreSQL (let's see what version 10 holds). PostgreSQL provides a solid, efficient SQL solution. Mongo provides no benefits of transactions or solid design. If you want either, look to Arango or PostgreSQL.

nir 9y ago

Marketing aside, MongoDB is just really easy to pick up and use. In a small project, its weaknesses don't really show up (at least in my limited experience), if it gets big you often end up rebuilding anyway.

chx 9y ago

For us, the reason to run with MongoDB in 2009 were twofold: a) for one of our products which was consuming the Twitter gardenhose, unverified writes were a blessing. We got away with a single dedicated server running mongodb (and it was not a particularly expensive server). I would need to do serious research again whether there's a better solution for this today. It was using Stanford NLP to highlight breaking news before anyone realized. We never got a single subscriber because every newsdesk that saw it wanted to acquire us immediately LOL. Eventually one did. b) For another, the ability to store and index complex documents which in SQL would've needed JOINs and you couldn't index across JOINs at least in MySQL and our previous experience in PostgreSQL vs MySQL support didn't favor PostgreSQL at the time at all. It worked -- although it didn't have a journal back then. We were ready for a failure but the failure never came. The landscape today is much different with PostgreSQL and MySQL and SQLite all embracing JSON storage and indexing on expressions are there for all three.

andred14 9y ago

Anyone with proper unbiased DB knowledge knows without trying or reading anything that it is hype. DBs are all about trade-offs you can't have it all

bastijn 9y ago

Obligatory read when talking MongoDB https://aphyr.com/posts/322-jepsen-mongodb-stale-reads https://aphyr.com/posts/322-jepsen-mongodb-stale-reads and follow-up https://jepsen.io/analyses/mongodb-3-4-0-rc3 https://jepsen.io/analyses/mongodb-3-4-0-rc3. Ensured me to never use this for production scenarios. No matter how 'hyped' or 'convenient' it is. Yaiks.

andred14 9y ago

Anyone with average to more DB experience knows there is no one-stop shop. DBs are about trade-offs and we need to choose the right ones for our application(s)

namlem 9y ago

Because they have great docs. At least that's why I chose them for my (now defunct) startup back in 2013. I wasn't familiar with databases and their docs were extremely approachable and easy to understand and navigate, with lots of useful examples.

nemild 9y ago

I wrote this series over several months and am happy to answer questions. Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr http://eepurl.com/cxs5Zr As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.

ahallock 9y ago

Marketing & ease of use (for example, if you try to insert into a collection that does not exist, it's automatically created for you).

jsherard 9y ago

I find it difficult to seriously address comments to this thread. People still referencing 10gen and talking about data points from 5-10 years ago (2009, 2012, 2013). You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft. http://bit.ly/2uP3Obi http://bit.ly/2uP3Obi MongoDB has many built in features that are extremely attractive for building modern applications. Built in HA & DR, horizontal scalability, multiple storage engines, rich indexing and query capabilities, enhanced BI features in aggregation framework, and many more...

logwriter 9y ago

You guys are discussing MongoDB from 2009. The database has evolved since then. - There is a new replication protocol for replica sets, improving the election process. - WiredTiger has been introduced and became the default storage engine. - Journaling guarantees durability. - Sharding architecture has changed. Config Servers are now deployed as Replica Sets and some of the tasks that used to be managed by mongos now are managed by the config servers. - Storage efficiency has been added through compression. - Security has been improved with encryption at rest. - The aggregation framework has more options for BI. - There are a variety of different indexes that can be created to improve query performance. - and finally, stop saying MongoDB is schemaless because it is not. There is a schema in place and this schema is flexible enough to be polyform. These are features available today in 3.4. You might have issues in the past, but if you really understand the newest architecture, you are capable of seeing how much MongoDB has grown since 2.4.

Daishiman 9y ago
But at this point that's all pretty much moot because competitors have caught up and there's hardly an advantage over Postgres' JSONB fields, which builds up on top of all the features Mongo will likely never have (transactions I'm looking at you).

felipeccastro 9y ago

It's not just because of hype that teams choose Mongo. The flexibility of a schemaless db is pretty helpful in a startup scenario, where the requirements drastically change all the time. Yes, migrations work for RDBMS too, but with Mongo you only write migrations for data changes, not schema changes - which means a lot less migrations. The option of having documents with embedded objects/lists makes it a pretty natural mapping to Domain Driven Design concepts, like aggregate roots + aggregates. One might even argue that while following DDD, it's preferable to handle most rules at the application layer and use the database simply as a performant, low-maintenance storage service. I know some folks here might view this approach as a sin, but it works pretty well too, especially for smaller teams without a dedicated DBA. Maybe it's just me that haven't ran large enough systems to find the kind of issues with mongo that people always rant about, idk. Honest question: considering all these advantages above (from a developer's point of view), are there good reasons for choosing something like PostgreSQL over Mongo for typical business apps?

busterarm 9y ago
I'm not entirely convinced that MongoDB is better suited or a natural mapping to DDD concepts in practice, it's just more convenient for developers who want to build things fast. My thoughts on this pattern are to still use a robust, reliable db (Postgres) behind a message queue for the event store. Postgres does almost all of the things I need with the reliability that I need while letting me sleep at night. An event store absolutely must be the most reliable part of the infrastructure, IMO.
- felipeccastro 9y ago
  "just more convenient for developers who want to build things fast" - that's a compelling advantage for Mongo, isn't it? I agree a robust, reliable db is super important - but isn't Mongo too, at this point? Even if it wasn't back in 2012? I thought most of the reliability issues and bad defaults were fixed by now...
  - busterarm 9y ago

nasalgoat 9y ago

We chose it because the lead developer liked it, and when I did my research I didn't find any glaring issues because it was too new. The lead dev had never actually used it, but liked the idea of not having to come up with a schema and he really liked JSON as he was very familiar with it. By the time we were done, we were easily one of the top 5 users of MongoDB at scale and it was responsible for 9 out of 10 of our outages due to various bugs or optimization issues.

ivm 9y ago

Our team had little experience with backends in 2013 and a fear that it will be hard to scale game servers in case of success (players hate lags and downtimes). So we followed the hype of Node.js and MongoDB because everybody was saying how it's easy to scale them compared to the established tech.

rfehrmann 9y ago

We are also one of those companies that bought into the promise (or hype) in 2011 and gave MongoDB a spin. After some very successful tests, I had long conversations with my CTO on why we should deploy MongoDB in prod and make it our primary OLTP store. Yes, MongoDB had it's challenges in 1.6 and it required some "faith". But it's 2017 now and MongoDB totally delivered on it’s vision, at least for us. Since 2012, that's when we upgraded to 2.0, we had 10 min of total downtime. That's 99.999% uptime in 5 years. Year after year, 99.999% uptime. And that 10 min is the total for scheduled (btw, we don't have scheduled downtime for our MongoDB clusters since there is no need for that) and un-scheduled downtime. We never take the service down. We perform all maintenance operations during regular business hours with no impact to end-users at all. So much about reliability! MongoDB as a product has bugs. What a surprise, what software product doesn't. But it's about you community (we are running the community edition) supports the product. We recently discovered an issue as well causing the primary to crash. As designed, another member in the repl set took over and there was no downtime at all. Even though we don't have a support contract, the issue was acknowledged by MongoDB in hours and fixed in a couple of days. I wish we had that type of support for products where we do have a support contract! And last but not least, if you can't figure out how to successfully use MongoDB, for god sake, take some free training classes on MongoDB University. MongoDB might not be a good fit for every use-case, but it's already an awesome platform for an incredible number of applications. Polyglot environments is the keyword here. SQL has it’s advantages and disadvantages and so does NoSQL. Use the best tool for the problem. One size fits all doesn't work anymore and MongoDB is clearly the best and most popular document store on the market. And even better, other platforms like postgres have acknowledged that the idea of storing JSON has its merits and have implemented its own version of that idea. However, IMO, if your use-case is a good fit for a document store, use MongoDB. We have done so and couldn't be happier with that decision.

jaraco 9y ago

I'd wanted to lift the burden of writing ORM layers and dealing with relational algebra and focus on persisting my intrinsic application models. I'd studied XML and other semi-structured web data at the graduate level, but was frustrated there wasn't a technology that was meeting that need. Companies like Microsoft and Oracle were adding XML functionality to their databases, but because it was bolted on an already constrained structure, it failed to address the mangling that relational structures impose. I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows. There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch? Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings. When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful. Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later. But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB. I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box. And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition. I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.