8 ms·

Out of curiosity, why did Mongo get so popular? I used it in university when it was hyped and it was pleasant to get started with. The ability to query your dat

Out of curiosity, why did Mongo get so popular? I used it in university when it was hyped and it was pleasant to get started with. The ability to query your data is so inconvenient though, I switched to SQL and I'm confused about what made experienced developers switch from SQL to mongo in the past. Is it just because you can scale up more easily when your database gets massive? Like NodeJS and many web technologies, almost everything I read about it years ago turned out to be hype and not based on facts (eg, nodejs being faster for large numbers of requests than traditional backends is not true, but it was widely repeated on the first 10 pages of google search results) edit: very well said, thanks for all the replies!

rapsey 5y ago
Because like NodeJS it has a low barrier to entry. This is by far the most important feature for rapid product uptake.
mcintyre1994 5y ago
I think a lot of it is how easy it is to insert, query and update nested objects compared to a normalised SQL database. I know that Postgres' JSON field has come on a lot in the mean time though.
mjburgess 5y ago
Rapid application development. During which, data is naturally hierarchical and schema-less (ie., not defined, changing). Mongo makes it easy just to "save your dictionary" It's really just an "sqlite for hashmaps", and should have remained this.
- fennecfoxen 5y ago
  More specifically, it's an mmap()ed series of linked lists of BSON documents, with some relatively simple B-tree indexing, a little journaling, and an unfortunate tendency to drop data on the floor.
  - bogomipz 5y ago
    Mongo hasn't used the MMAP storage engine for a long time. In fact it looks like it has even been removed now: https://docs.mongodb.com/manual/core/storage-engines/ https://docs.mongodb.com/manual/core/storage-engines/ Wired Tiger, the default storage engine is LSM Trees with BTrees for indexes.
    - matchy 5y ago
      Not LSM, all BTrees, but the rest is correct.
      - bogomipz 5y ago
        Not sure of the current state of Mongo but WT certainly has LSM Trees: http://www.hpts.ws/papers/2013/WiredTiger-HPTS.pdf http://www.hpts.ws/papers/2013/WiredTiger-HPTS.pdf http://source.wiredtiger.com/3.2.1/lsm.html http://source.wiredtiger.com/3.2.1/lsm.html
flowerlad 5y ago
SQL relies on joins. Joins are fine for reasonably sized databases, but on internet scale joins have miserable performance. "Document" based databases such as Mongo are designed to avoid joins. MongoDB has automatic sharding, which is very important for horizontal scaling of writes. For MySQL you have to do manual sharding which is extremely hard. And finally, document oriented databases are schemaless, which means there is no downtime when you add/remove fields.
- gbuk2013 5y ago
  That’s the theory, in practice “no schema” in my recent painful experience translates to “move the schema into the apps” which really sucks when you have a bunch of interdependent services owned by different teams. And you still need to do joins to enrich data from different collections. I want Postgres back. :(
  - flowerlad 5y ago
    > really sucks when you have a bunch of interdependent services owned by different teams. That's true. The strictness of SQL lets you use SQL as the integration point. Without it you'd have to write a service on top as the integration layer.
  - laichzeit0 5y ago
    It’s exactly that. Shifting the problem from schema-on-write to schema-on-read. It’s like dynamic vs statically typed. Personally, I don’t like it at all, none of your problems have magically disappeared, you just shifted it to runtime (or the application layer, as you say).
    - vel0city 5y ago
      In many document databases these days you can have required fields and types enforced by the database to ensure all documents in the collection have at least a certain schema. https://docs.mongodb.com/manual/core/schema-validation/#restrictions https://docs.mongodb.com/manual/core/schema-validation/#rest...
ithrow 5y ago
IMO, nodejs today is a reliable tech for many different kinds of server side workloads, I wouldn't compare it to mongodb.
- gentleman11 5y ago
  I wouldn't say mongo is unreliable, and I enjoy node a lot. Its just that, years ago, everybody said it was amazing for certain things and it wasn't based on anything
MildlySerious 5y ago
Mongo popped up around the time node started growing more in popularity. Around 2012, 2013 I started hearing it everywhere, and it felt like a hard pressed marketing attempt to make Mongo the database for node, and by extension the database for the internet. The web-scale meme came from them actually trying to position themselves like that, and it partially worked given that it is not only still around, but the prevalence of the MEAN/MERN stack, at least in (tech) pop culture and social media. To this day my opinion is that Mongo is a snake oil PR firm with a "database" and I have not yet seen anything that has convinced me otherwise.
tayloramurphy 5y ago
Reading why RethinkDB failed may provide additional context as to how Mongo was successful. https://www.defmacro.org/2017/01/18/why-rethinkdb-failed.html https://www.defmacro.org/2017/01/18/why-rethinkdb-failed.htm...
gbuk2013 5y ago
(Just finished their M121 course for my newish job). Their aggregation pipelines are pretty pleasant to use. You basically get a bunch of data transformation code that you can avoid writing and you can work with your data without it going out on the wire. I would probably be able to do most of that in SQL and I’m sure someone who really knows SQL can do everything it can do but I suspect the query involved will be significantly gnarlier. :) Not sure about how the two would compare performance wise but I do know that you can’t use “$lookup” (Mongo equivalent of joins) with sharding which strikes be as quite unfortunate for scalability.
vel0city 5y ago
You don't use a screwdriver to drive a nail, and you don't user a hammer to drive a screw. Its good to use the right tool for the job. In many instances, a relational database (often SQL) makes a lot of sense and is the right tool for the job. Some workflows, a NoSQL/Non-relational database design can be nice. Document databases, K/V stores, etc can be more optimized for those specific use cases which can lead to some better experiences. That said, lots of relational databases are quite powerful and quite performant that unless you really need the particular use cases the NoSQL databases focus on, it may just be better to use the tooling available in your relational database. There are a few instances where I've really preferred using a document database. Things like logs are often really useful to have in a document database like MongoDB. In one collection I can stick all kinds of logs. The schema of all the different fields in the document are really relevant to what kind of things I'm logging. If I were to have a column for every field I might have in the log message, each row would have a ton of null columns. If I wanted to add a new field to easily search on, I might have to add a new column. A "schemaless" design can then be very useful. Of course this is true of pretty much any document database and not necessarily only MongoDB, and these days some relational databases have decent JSON document database functionality as well. There are always certain kinds of tradeoffs to be made when choosing one database technology over another. Another thing I've found useful with MongoDB in particular is GridFS, that has been pretty neat to use. These days there are other systems of handling binary blob storage, but as mentioned earlier there are always tradeoffs with choosing any particular technology.
whydid 5y ago
Junior developers are terrified of their application running slow, and not having the technical aptitude required to debug and fix the performance problem. Additionally, Junior developers do not care about data integrity (the customer's problem), or operational complexity (the SRE's problem). Finally, Junior developers consider software to be much like bread. Older software is bad (such as SQL relational databases), and newer software is good (NoSQL). Combine all three of these flawed systems of thinking, and you have MongoDB adaption. ... also before Kafka, the defacto queue broker was ActiveMQ, which was a pain to deploy, so many people used Mongo as a queue.
bogomipz 5y ago
I believe it became popular due to the following: It had a flexible data model as there is no predefined schema. The selling point was rapid protoyping. The JSON-based documents meant developers could work in objects without an ORM. If I remember Mongo called this "fixing the data impedence mismatch." Sigh. It had decent performance as long as your working set fit in RAM as it was basically a mmap()'d file. It was easy to get up and running. It was in the right place at the right time during the original NOSQL hype cycle.

rapsey 5y ago

Because like NodeJS it has a low barrier to entry. This is by far the most important feature for rapid product uptake.

mcintyre1994 5y ago

I think a lot of it is how easy it is to insert, query and update nested objects compared to a normalised SQL database. I know that Postgres' JSON field has come on a lot in the mean time though.

mjburgess 5y ago

Rapid application development. During which, data is naturally hierarchical and schema-less (ie., not defined, changing). Mongo makes it easy just to "save your dictionary" It's really just an "sqlite for hashmaps", and should have remained this.

fennecfoxen 5y ago
More specifically, it's an mmap()ed series of linked lists of BSON documents, with some relatively simple B-tree indexing, a little journaling, and an unfortunate tendency to drop data on the floor.
- bogomipz 5y ago
  Mongo hasn't used the MMAP storage engine for a long time. In fact it looks like it has even been removed now: https://docs.mongodb.com/manual/core/storage-engines/ https://docs.mongodb.com/manual/core/storage-engines/ Wired Tiger, the default storage engine is LSM Trees with BTrees for indexes.
  - matchy 5y ago
    Not LSM, all BTrees, but the rest is correct.
    - bogomipz 5y ago
      Not sure of the current state of Mongo but WT certainly has LSM Trees: http://www.hpts.ws/papers/2013/WiredTiger-HPTS.pdf http://www.hpts.ws/papers/2013/WiredTiger-HPTS.pdf http://source.wiredtiger.com/3.2.1/lsm.html http://source.wiredtiger.com/3.2.1/lsm.html

flowerlad 5y ago

SQL relies on joins. Joins are fine for reasonably sized databases, but on internet scale joins have miserable performance. "Document" based databases such as Mongo are designed to avoid joins. MongoDB has automatic sharding, which is very important for horizontal scaling of writes. For MySQL you have to do manual sharding which is extremely hard. And finally, document oriented databases are schemaless, which means there is no downtime when you add/remove fields.

gbuk2013 5y ago
That’s the theory, in practice “no schema” in my recent painful experience translates to “move the schema into the apps” which really sucks when you have a bunch of interdependent services owned by different teams. And you still need to do joins to enrich data from different collections. I want Postgres back. :(
- flowerlad 5y ago
  > really sucks when you have a bunch of interdependent services owned by different teams. That's true. The strictness of SQL lets you use SQL as the integration point. Without it you'd have to write a service on top as the integration layer.
- laichzeit0 5y ago
  It’s exactly that. Shifting the problem from schema-on-write to schema-on-read. It’s like dynamic vs statically typed. Personally, I don’t like it at all, none of your problems have magically disappeared, you just shifted it to runtime (or the application layer, as you say).
  - vel0city 5y ago
    In many document databases these days you can have required fields and types enforced by the database to ensure all documents in the collection have at least a certain schema. https://docs.mongodb.com/manual/core/schema-validation/#restrictions https://docs.mongodb.com/manual/core/schema-validation/#rest...

ithrow 5y ago

IMO, nodejs today is a reliable tech for many different kinds of server side workloads, I wouldn't compare it to mongodb.

gentleman11 5y ago
I wouldn't say mongo is unreliable, and I enjoy node a lot. Its just that, years ago, everybody said it was amazing for certain things and it wasn't based on anything

MildlySerious 5y ago

Mongo popped up around the time node started growing more in popularity. Around 2012, 2013 I started hearing it everywhere, and it felt like a hard pressed marketing attempt to make Mongo the database for node, and by extension the database for the internet. The web-scale meme came from them actually trying to position themselves like that, and it partially worked given that it is not only still around, but the prevalence of the MEAN/MERN stack, at least in (tech) pop culture and social media. To this day my opinion is that Mongo is a snake oil PR firm with a "database" and I have not yet seen anything that has convinced me otherwise.

tayloramurphy 5y ago

Reading why RethinkDB failed may provide additional context as to how Mongo was successful. https://www.defmacro.org/2017/01/18/why-rethinkdb-failed.html https://www.defmacro.org/2017/01/18/why-rethinkdb-failed.htm...

gbuk2013 5y ago

(Just finished their M121 course for my newish job). Their aggregation pipelines are pretty pleasant to use. You basically get a bunch of data transformation code that you can avoid writing and you can work with your data without it going out on the wire. I would probably be able to do most of that in SQL and I’m sure someone who really knows SQL can do everything it can do but I suspect the query involved will be significantly gnarlier. :) Not sure about how the two would compare performance wise but I do know that you can’t use “$lookup” (Mongo equivalent of joins) with sharding which strikes be as quite unfortunate for scalability.

vel0city 5y ago

You don't use a screwdriver to drive a nail, and you don't user a hammer to drive a screw. Its good to use the right tool for the job. In many instances, a relational database (often SQL) makes a lot of sense and is the right tool for the job. Some workflows, a NoSQL/Non-relational database design can be nice. Document databases, K/V stores, etc can be more optimized for those specific use cases which can lead to some better experiences. That said, lots of relational databases are quite powerful and quite performant that unless you really need the particular use cases the NoSQL databases focus on, it may just be better to use the tooling available in your relational database. There are a few instances where I've really preferred using a document database. Things like logs are often really useful to have in a document database like MongoDB. In one collection I can stick all kinds of logs. The schema of all the different fields in the document are really relevant to what kind of things I'm logging. If I were to have a column for every field I might have in the log message, each row would have a ton of null columns. If I wanted to add a new field to easily search on, I might have to add a new column. A "schemaless" design can then be very useful. Of course this is true of pretty much any document database and not necessarily only MongoDB, and these days some relational databases have decent JSON document database functionality as well. There are always certain kinds of tradeoffs to be made when choosing one database technology over another. Another thing I've found useful with MongoDB in particular is GridFS, that has been pretty neat to use. These days there are other systems of handling binary blob storage, but as mentioned earlier there are always tradeoffs with choosing any particular technology.

whydid 5y ago

Junior developers are terrified of their application running slow, and not having the technical aptitude required to debug and fix the performance problem. Additionally, Junior developers do not care about data integrity (the customer's problem), or operational complexity (the SRE's problem). Finally, Junior developers consider software to be much like bread. Older software is bad (such as SQL relational databases), and newer software is good (NoSQL). Combine all three of these flawed systems of thinking, and you have MongoDB adaption. ... also before Kafka, the defacto queue broker was ActiveMQ, which was a pain to deploy, so many people used Mongo as a queue.

bogomipz 5y ago

I believe it became popular due to the following: It had a flexible data model as there is no predefined schema. The selling point was rapid protoyping. The JSON-based documents meant developers could work in objects without an ORM. If I remember Mongo called this "fixing the data impedence mismatch." Sigh. It had decent performance as long as your working set fit in RAM as it was basically a mmap()'d file. It was easy to get up and running. It was in the right place at the right time during the original NOSQL hype cycle.