6 ms·

Yes. If we were going to start from scratch today, we'd probably use Postgres. But, realistically, the primary motivation behind that decision would be because

by 013a 6y ago

Yes. If we were going to start from scratch today, we'd probably use Postgres. But, realistically, the primary motivation behind that decision would be because Postgres is available on AWS, and that would centralize more of our operations. (DocumentDB is, of course available. Its not Mongo. I'd be curious to hear from people who actually had Mongo deployments and were able to move to DocumentDB; its missing so many of MongoDB's APIs that we physically can't, our applications would not run). Mongo isn't that bad. It has limitations. You work within the limitations... or you don't. But I really don't think a valid option is "mongodb fuckin sucks m8, shit tier db". We're not going to be migrating terabytes of data and tens of thousands of lines of code when the benefit is tenuous for our business domain. Should you use MongoDB today? I'll say No, but not for the reasons anyone else is giving. MongoDB's Cloud Provider agreement has decimated the cloud marketplace for the database. Essentially, if you want to run a version released in the past few years (4.2+), you need to be on their first-party Atlas product. Many other parties, especially the big clouds, are on 3.6 (or have compatibility products like DocumentDB/CosmosDB which target 3.6). Atlas is great. Its fairly priced and has a great UX and operations experience. But, I don't feel comfortable about there being political reasons why I couldn't change providers if that changes. If you have business requirements which demand, say, data in a specific region, or government-class infra, or specific compliance frameworks, Atlas may not be able to meet them.

mbell 6y ago
> We're not going to be migrating terabytes of data You may have dramatically less 'real data' than mongo makes you think you do. I migrated one of our mid sized database out of mongo and into PG a couple years ago. The reduction in size was massive. One table in particular that was storing a small number of numeric fields per doc went from ~10GB to ~50MB. I wouldn't expect this with all datasets of course, but mongo's document + key storage overhead can be massive in some use cases.
- damidekronik 6y ago
  Not OP but I think it's more about the importance of the said data, number of collections to think about and so on. Regarding your point, I would guess some index changes might have had a significant impact here.
  - mbell 6y ago
    > Regarding your point, I would guess some index changes might have had a significant impact here. I'm not sure exactly what you mean, but that particular collection only had a single index on it (outside the ID column).
- svachalek 6y ago
  Unless this has changed in recent years, the BSON format that Mongo uses is more or less JSON optimized for parsing speed and takes more or less as much space as storing your entire database in JSON. JSON is a great format for simplicity and readability but as a storage format it's hard to come up with one that's more bloated.
  - yetihehe 6y ago
    > but as a storage format it's hard to come up with one that's more bloated. It's easy: xml.
    - jonnypotty 6y ago
      Work in the book industry, our implementation of it(onix) is a decent way to allow non IT professionals to encode complex data in a standardised way, but as a way to store and transmit large amounts of data its a nightmare. The only thing that saves it is there is so much repeated data it compresses brilliantly. Ha.
- thomascgalvin 6y ago
  This is (probably) an artifact of Mongo's schema-less nature; when you don't have tables with structure, every document you store has to detail its own schema inline. In a relational database, you have columns with names and types, and that info is shared by all of the rows. In Mongo, every cell has to specify its name and type, even if that layout is shared by every other cell in the document. Mongo's way is more flexible, but it's terrible for storage efficiency.
  - rubyfan 6y ago
    /caveat i know nothing about how mongo is storing data and haven’t used it since 2010 it doesn’t really have to approach the schemas that way. one would think it would be optimized for repeat schema in the same way one might create the schema definition and then reference it in packing and unpacking the data. seems like if there was schema overhead taking up storage unnecessarily that could be optimized relatively easily. having schema references also might be a good management tool to understand which records vary potentially due to an application evolving it’s needs.
    - carlps 6y ago
      There's something beautiful about normalizing the storage of denormalized schemas.
      - jgalt212 6y ago
        Yes, and the sooner you do it the better. But doing it during project planning / experimentation phase (or when you don't know what the final result should be yet) will really just slow you down. In many ways, very similar to the static / dynamic language trade offs.
    - thomascgalvin 6y ago
      > one would think it would be optimized for repeat schema I don't think that's the problem Mongo is designed to solve. Mongo's promise was the ability to work with un- and semi- structured data, and it would make sense if its optimizations were focused on that problem, not on reducing overhead when someone tries to strongarm it into being MySQL. Generally speaking, if your data is structured well enough that you can define a schema ahead of time, you're better off with a traditional RDBMS, because that's the problem an RDBMS is designed to solve.
gigatexal 6y ago
I actually love the idea of providers that abstract the major cloud vendors to run things like Mongo does with Atlas: you can spin up Mongo on any of the three -- boom lock-in concerns gone, and the best part, is you're both supporting the project but also have the creators for tech-support.
collyw 6y ago
I'll give you a reason that not many people mention not to use Mongo. Schema definition acts as a form of documentation for your database. As someone that has come into a legacy project built on Mongo, its a nightmare trying to work out the structure of the database. Especially as there is redundant copies of some data in different collections.
gorkemcetin 6y ago
Regarding your question about the ability to be able to move to DocumentDB from MongoDB, we (Countly Analytics team) weren't able since several APIs in newer MongoDB releases are still not available in DocumentDB.

by 013a 6y ago

mbell 6y ago

> We're not going to be migrating terabytes of data You may have dramatically less 'real data' than mongo makes you think you do. I migrated one of our mid sized database out of mongo and into PG a couple years ago. The reduction in size was massive. One table in particular that was storing a small number of numeric fields per doc went from ~10GB to ~50MB. I wouldn't expect this with all datasets of course, but mongo's document + key storage overhead can be massive in some use cases.

damidekronik 6y ago
Not OP but I think it's more about the importance of the said data, number of collections to think about and so on. Regarding your point, I would guess some index changes might have had a significant impact here.
- mbell 6y ago
  > Regarding your point, I would guess some index changes might have had a significant impact here. I'm not sure exactly what you mean, but that particular collection only had a single index on it (outside the ID column).
svachalek 6y ago
Unless this has changed in recent years, the BSON format that Mongo uses is more or less JSON optimized for parsing speed and takes more or less as much space as storing your entire database in JSON. JSON is a great format for simplicity and readability but as a storage format it's hard to come up with one that's more bloated.
- yetihehe 6y ago
  > but as a storage format it's hard to come up with one that's more bloated. It's easy: xml.
  - jonnypotty 6y ago
    Work in the book industry, our implementation of it(onix) is a decent way to allow non IT professionals to encode complex data in a standardised way, but as a way to store and transmit large amounts of data its a nightmare. The only thing that saves it is there is so much repeated data it compresses brilliantly. Ha.
thomascgalvin 6y ago
This is (probably) an artifact of Mongo's schema-less nature; when you don't have tables with structure, every document you store has to detail its own schema inline. In a relational database, you have columns with names and types, and that info is shared by all of the rows. In Mongo, every cell has to specify its name and type, even if that layout is shared by every other cell in the document. Mongo's way is more flexible, but it's terrible for storage efficiency.
- rubyfan 6y ago
  /caveat i know nothing about how mongo is storing data and haven’t used it since 2010 it doesn’t really have to approach the schemas that way. one would think it would be optimized for repeat schema in the same way one might create the schema definition and then reference it in packing and unpacking the data. seems like if there was schema overhead taking up storage unnecessarily that could be optimized relatively easily. having schema references also might be a good management tool to understand which records vary potentially due to an application evolving it’s needs.
  - carlps 6y ago
    There's something beautiful about normalizing the storage of denormalized schemas.
    - jgalt212 6y ago
      Yes, and the sooner you do it the better. But doing it during project planning / experimentation phase (or when you don't know what the final result should be yet) will really just slow you down. In many ways, very similar to the static / dynamic language trade offs.
  - thomascgalvin 6y ago
    > one would think it would be optimized for repeat schema I don't think that's the problem Mongo is designed to solve. Mongo's promise was the ability to work with un- and semi- structured data, and it would make sense if its optimizations were focused on that problem, not on reducing overhead when someone tries to strongarm it into being MySQL. Generally speaking, if your data is structured well enough that you can define a schema ahead of time, you're better off with a traditional RDBMS, because that's the problem an RDBMS is designed to solve.

gigatexal 6y ago

I actually love the idea of providers that abstract the major cloud vendors to run things like Mongo does with Atlas: you can spin up Mongo on any of the three -- boom lock-in concerns gone, and the best part, is you're both supporting the project but also have the creators for tech-support.

collyw 6y ago

I'll give you a reason that not many people mention not to use Mongo. Schema definition acts as a form of documentation for your database. As someone that has come into a legacy project built on Mongo, its a nightmare trying to work out the structure of the database. Especially as there is redundant copies of some data in different collections.

gorkemcetin 6y ago

Regarding your question about the ability to be able to move to DocumentDB from MongoDB, we (Countly Analytics team) weren't able since several APIs in newer MongoDB releases are still not available in DocumentDB.