6 ms·

We currently use MongoDB and while Postgres is attractive for so many reasons, even with Amazon Aurora's Postgres we still need legacy "database maintenance win

We currently use MongoDB and while Postgres is attractive for so many reasons, even with Amazon Aurora's Postgres we still need legacy "database maintenance windows" in order to achieve major version upgrades. With MongoDB, you're guaranteed single-prior-version replication compatibility within a cluster. This means you spin up an instance with the updated version of MongoDB, it catches up to the cluster. Zero downtime, seamless transition. There may be less than a handful of cancelled queries that are retryable but no loss of writes with their retryable writes and write concern preferences. e.g. MongoDB 3.6 can be upgraded to MongoDB 4.0 without downtime. Edit: Possibly misinformed but the last deep dive we did indicated there was not a way to use logical replication for seamless upgrades. Will have to research.

deleted 5y ago
[deleted]
TedShiller 5y ago
The problem with MongoDB though is that you're on MongoDB
- salil999 5y ago
  And how exactly is that a problem?
  - mayankkaizen 5y ago
    That was sarcasm. But yeah, you can search for MongoDB and you'll come across many many posts criticizing it. It can be said Mongodb is hated ad much as Postgres is loved. Personally I have no opinion about mongodb.
    - The_Colonel 5y ago
      Yes, and most of these love/hate memes are blowned out of proportion by people who don't actually have any real expertise in those technologies, but just parrot whatever they've read in some memes.
      - staticassertion 5y ago
        You're exactly correct. Tons of "XYZ is bad" because of some meme that they don't even understand or have context on that hasn't been relevant for years. I have no idea if MongoDB is good or bad at this point, but the comments of "haha it's mongo" are completely devoid of meaningful content and should be flagged.
        operator9A 5y ago
        I was part of a team that operated a large Mongo cluster for most of the last decade. I would not have advised anyone to use Mongo as their durable source of truth database then, and I still don't think it's advisable to do so now. On numerous occasions, Mongo demonstrated the consequences of poor engineering judgment and an addled approach to logic in critical components responsible for data integrity. In addition, Mongo internalized many poor patterns with respect to performance and change management. Mongo did not, and does not provide the data integrity or performance guarantees that other databases internalize by design (the WiredTiger transition helped, but did not cure many of the issues). PostgreSQL introduced JSONB GIN index support sometime around 2015, making Postgres a better fit for most JSON-based applications than Mongo.
        staticassertion 5y ago
        My issue isn't with people not liking Mongo. It's with contentless meme posts. Your post has real information that adds value to the conversation, and I appreciate that you took the time to write it out.
        cztomsik 5y ago
        Because it's (or at least it definitely WAS) true. There are valid use-cases for mongo but for vast majority of things, you're better to start with postgres. And I say that as an early adopter - I really wanted mongo to succeed but it just failed all of my expectations. All of them. BTW: this is post about postgres.
    - threeseed 5y ago
      You can find posts criticising every database. Most of the ones for MongoDB are from a decade ago and not at all relevant today.
  - tpxl 5y ago
    Some things seem to have changed from 2018, but MongoDB was by far the worst database I ever had the displeasure of using (and Amazon DocumentDB was even worse). https://jepsen.io/analyses/mongodb-3-6-4 https://jepsen.io/analyses/mongodb-3-6-4 https://jepsen.io/analyses/mongodb-4.2.6 https://jepsen.io/analyses/mongodb-4.2.6
    - jd_mongodb 5y ago
      Posting old Jepssen analyses is like pointing at old bug reports. Everytime Jepsen finds a bug we fix it lickety-split. I know it's not cool to focus on that fact, but it is a fact. The Jepsen tests are part of the MongoDB test suite so when we fix those problems they stay fixed. I would love to hear your personal experience of MongoDB as opposed to reposting old Jepsen reports. Perhaps there is something that we can address in 5.1 that is still a problem? (I work in developer relations at MongoDB)
      - sigmonsays 5y ago
        where is the latest jepsen test results published?
      - jenny91 5y ago
        The latest "old Jepsen report" is barely a year old. It's not like digging up dirt from years ago. It also seems like there was quite a lot wrong even a year ago, quoting from there: > Roughly 10% of transactions exhibited anomalies during normal operation, without faults. It's just not a very reassuring response to say "when someone goes to dig a bit and finds a lot of show-stopping bugs, we address those specific bugs quickly". To me it sounds like the architecture and care just isn't there for a robust data storage layer?
        jiggawatts 5y ago
        Something that was drilled into me decades ago is that there is no such thing as fixing multi-threaded (or distributed) code via debugging or patching it "until it works". You either mathematically prove that it is correct, or it is wrong for certain. This sounds like an oddly strong statement to say, but the guy who wrote the textbook that contained that statement went on to dig up trivial looking examples from other textbooks that were subtly wrong. His more qualified statement is that if a professor writing simplified cases in textbooks can't get it right, then the overworked developer under time pressure writing something very complex has effectively zero chance. The MongoDB guys just don't understand this. They're convinced that if they plug just one more hole in the wire mesh, then it'll be good enough for their submarine. PS: The professor I was referring to is Doug Lea, who wrote the "EDU.oswego.cs.dl.util.concurrent" library for Java. This was then used as the basis for the official "java.util.concurrent".
  - TedShiller 5y ago
    It's easier to ask how that is NOT a problem, because that list will be much, much shorter.
- kbenson 5y ago
  As someone that mostly shared that opinion for the last decade or more, I recently set up a cluster for work, and everything seems much more production level quality than I remember or what I assumed it was going to be like. I'm not the one using it for queries every day, but I did do a bunch of testing for replication and failed nodes to confirm that I understood (and could rely) on the claims of robustness, and it seemed to be stable and with good documentation of what to expect in different scenarios and how to configure it (which is not what I experienced doing the same testing back in 2010-2011). All in all, my impression of MongoDB now is that they're one of those "fake it till you make it" success stories, where they leveraged their popularity into enough momentum to fix most their major problems.
  - outworlder 5y ago
    One thing that turned me away from MongoDB was their utter lack of care for your data integrity that they displayed for years. Some of those instances were even documented. Then there were some bad defaults - some could _also_ cause data loss. For any component that's viewed as a database (as opposed to, say, cache), data integrity is one of the most important metrics (if not THE most). In contrast, PostgreSQL data loss bugs are rare - and are treated extremely seriously. Defaults are sane and won't lose data. It's one of the few databases I'm pretty confident that data will be there even if you yank a server power cord mid writes. Has MongoDB improved? Yes, leaps and bounds(seems to still fail Jepsen tests though). But I can't help but feel that it should have been released as a beta product, instead of claiming it was production ready. It wasn't. Maybe it is now. I'd still evaluate other alternatives before considering it. That said, one thing that always amuses me is how MongoDB gets mentioned in the same context as PostgreSQL. If PostgreSQL would meet your needs, it's unlikely that MongoDB would. And vice-versa(but maybe something else like Cassandra would).
    - threeseed 5y ago
      Those defaults were changed a decade ago and were never an issue if you used a driver eg. Python. And the Jepsen tests are part of the core test suite so do you some evidence they are still failing. It’s so ridiculous and pointless to be rehashing the same issues a decade later. Actually more a testament to the company that it’s still hugely successful and depended on by some very large applications.
ggregoire 5y ago
MongoDB and Postgres are like apples and oranges tho. I'm not gonna choose MongoDB if I need a relational model… even if it offers zero downtime upgrades out-of-the-box.
- bmcahren 5y ago
  You might choose Postgres with JSON as an alternative to MongoDB though. There are plenty of people pushing the limits of MongoDB who are researching it if not just for access to a larger pool of DBAs who can work for them.
zozbot234 5y ago
Logical replication across major releases for seamless upgrades has been supported and documented since pgSQL 10.
outworlder 5y ago
> Edit: Possibly misinformed but the last deep dive we did indicated there was not a way to use logical replication for seamless upgrades. Will have to research. It is possible since PG10 https://severalnines.com/database-blog/how-upgrade-postgresql10-postgresql11-zero-downtime https://severalnines.com/database-blog/how-upgrade-postgresq...
- bmcahren 5y ago
  We seem to have been misguided by all of the Amazon RDS and Aurora documentation. It seems Amazon prefers to implement postgres logical replication through their database migration service. All upgrades are typically done through pg_upgrade which does require downtime. Interesting. I can't wait to see how PG12 influences future offerings from the cloud providers for more seamless major version upgrades.

deleted 5y ago

[deleted]

TedShiller 5y ago

The problem with MongoDB though is that you're on MongoDB

salil999 5y ago
And how exactly is that a problem?
- mayankkaizen 5y ago
  That was sarcasm. But yeah, you can search for MongoDB and you'll come across many many posts criticizing it. It can be said Mongodb is hated ad much as Postgres is loved. Personally I have no opinion about mongodb.
  - The_Colonel 5y ago
    Yes, and most of these love/hate memes are blowned out of proportion by people who don't actually have any real expertise in those technologies, but just parrot whatever they've read in some memes.
    - staticassertion 5y ago
      You're exactly correct. Tons of "XYZ is bad" because of some meme that they don't even understand or have context on that hasn't been relevant for years. I have no idea if MongoDB is good or bad at this point, but the comments of "haha it's mongo" are completely devoid of meaningful content and should be flagged.
      - operator9A 5y ago
        I was part of a team that operated a large Mongo cluster for most of the last decade. I would not have advised anyone to use Mongo as their durable source of truth database then, and I still don't think it's advisable to do so now. On numerous occasions, Mongo demonstrated the consequences of poor engineering judgment and an addled approach to logic in critical components responsible for data integrity. In addition, Mongo internalized many poor patterns with respect to performance and change management. Mongo did not, and does not provide the data integrity or performance guarantees that other databases internalize by design (the WiredTiger transition helped, but did not cure many of the issues). PostgreSQL introduced JSONB GIN index support sometime around 2015, making Postgres a better fit for most JSON-based applications than Mongo.
        staticassertion 5y ago
        My issue isn't with people not liking Mongo. It's with contentless meme posts. Your post has real information that adds value to the conversation, and I appreciate that you took the time to write it out.
      - cztomsik 5y ago
        Because it's (or at least it definitely WAS) true. There are valid use-cases for mongo but for vast majority of things, you're better to start with postgres. And I say that as an early adopter - I really wanted mongo to succeed but it just failed all of my expectations. All of them. BTW: this is post about postgres.
  - threeseed 5y ago
    You can find posts criticising every database. Most of the ones for MongoDB are from a decade ago and not at all relevant today.
- tpxl 5y ago
  Some things seem to have changed from 2018, but MongoDB was by far the worst database I ever had the displeasure of using (and Amazon DocumentDB was even worse). https://jepsen.io/analyses/mongodb-3-6-4 https://jepsen.io/analyses/mongodb-3-6-4 https://jepsen.io/analyses/mongodb-4.2.6 https://jepsen.io/analyses/mongodb-4.2.6
  - jd_mongodb 5y ago
    Posting old Jepssen analyses is like pointing at old bug reports. Everytime Jepsen finds a bug we fix it lickety-split. I know it's not cool to focus on that fact, but it is a fact. The Jepsen tests are part of the MongoDB test suite so when we fix those problems they stay fixed. I would love to hear your personal experience of MongoDB as opposed to reposting old Jepsen reports. Perhaps there is something that we can address in 5.1 that is still a problem? (I work in developer relations at MongoDB)
    - sigmonsays 5y ago
      where is the latest jepsen test results published?
    - jenny91 5y ago
      The latest "old Jepsen report" is barely a year old. It's not like digging up dirt from years ago. It also seems like there was quite a lot wrong even a year ago, quoting from there: > Roughly 10% of transactions exhibited anomalies during normal operation, without faults. It's just not a very reassuring response to say "when someone goes to dig a bit and finds a lot of show-stopping bugs, we address those specific bugs quickly". To me it sounds like the architecture and care just isn't there for a robust data storage layer?
      - jiggawatts 5y ago
        Something that was drilled into me decades ago is that there is no such thing as fixing multi-threaded (or distributed) code via debugging or patching it "until it works". You either mathematically prove that it is correct, or it is wrong for certain. This sounds like an oddly strong statement to say, but the guy who wrote the textbook that contained that statement went on to dig up trivial looking examples from other textbooks that were subtly wrong. His more qualified statement is that if a professor writing simplified cases in textbooks can't get it right, then the overworked developer under time pressure writing something very complex has effectively zero chance. The MongoDB guys just don't understand this. They're convinced that if they plug just one more hole in the wire mesh, then it'll be good enough for their submarine. PS: The professor I was referring to is Doug Lea, who wrote the "EDU.oswego.cs.dl.util.concurrent" library for Java. This was then used as the basis for the official "java.util.concurrent".
- TedShiller 5y ago
  It's easier to ask how that is NOT a problem, because that list will be much, much shorter.
kbenson 5y ago
As someone that mostly shared that opinion for the last decade or more, I recently set up a cluster for work, and everything seems much more production level quality than I remember or what I assumed it was going to be like. I'm not the one using it for queries every day, but I did do a bunch of testing for replication and failed nodes to confirm that I understood (and could rely) on the claims of robustness, and it seemed to be stable and with good documentation of what to expect in different scenarios and how to configure it (which is not what I experienced doing the same testing back in 2010-2011). All in all, my impression of MongoDB now is that they're one of those "fake it till you make it" success stories, where they leveraged their popularity into enough momentum to fix most their major problems.
- outworlder 5y ago
  One thing that turned me away from MongoDB was their utter lack of care for your data integrity that they displayed for years. Some of those instances were even documented. Then there were some bad defaults - some could _also_ cause data loss. For any component that's viewed as a database (as opposed to, say, cache), data integrity is one of the most important metrics (if not THE most). In contrast, PostgreSQL data loss bugs are rare - and are treated extremely seriously. Defaults are sane and won't lose data. It's one of the few databases I'm pretty confident that data will be there even if you yank a server power cord mid writes. Has MongoDB improved? Yes, leaps and bounds(seems to still fail Jepsen tests though). But I can't help but feel that it should have been released as a beta product, instead of claiming it was production ready. It wasn't. Maybe it is now. I'd still evaluate other alternatives before considering it. That said, one thing that always amuses me is how MongoDB gets mentioned in the same context as PostgreSQL. If PostgreSQL would meet your needs, it's unlikely that MongoDB would. And vice-versa(but maybe something else like Cassandra would).
  - threeseed 5y ago
    Those defaults were changed a decade ago and were never an issue if you used a driver eg. Python. And the Jepsen tests are part of the core test suite so do you some evidence they are still failing. It’s so ridiculous and pointless to be rehashing the same issues a decade later. Actually more a testament to the company that it’s still hugely successful and depended on by some very large applications.

ggregoire 5y ago

MongoDB and Postgres are like apples and oranges tho. I'm not gonna choose MongoDB if I need a relational model… even if it offers zero downtime upgrades out-of-the-box.

bmcahren 5y ago
You might choose Postgres with JSON as an alternative to MongoDB though. There are plenty of people pushing the limits of MongoDB who are researching it if not just for access to a larger pool of DBAs who can work for them.

zozbot234 5y ago

Logical replication across major releases for seamless upgrades has been supported and documented since pgSQL 10.

outworlder 5y ago

> Edit: Possibly misinformed but the last deep dive we did indicated there was not a way to use logical replication for seamless upgrades. Will have to research. It is possible since PG10 https://severalnines.com/database-blog/how-upgrade-postgresql10-postgresql11-zero-downtime https://severalnines.com/database-blog/how-upgrade-postgresq...

bmcahren 5y ago
We seem to have been misguided by all of the Amazon RDS and Aurora documentation. It seems Amazon prefers to implement postgres logical replication through their database migration service. All upgrades are typically done through pg_upgrade which does require downtime. Interesting. I can't wait to see how PG12 influences future offerings from the cloud providers for more seamless major version upgrades.