3 ms·

Epic had a post mortem blog post here that mentioned in passing they had stumped all the experts they could find to look at unsolveable issues they had with Mon

Epic had a post mortem blog post here that mentioned in passing they had stumped all the experts they could find to look at unsolveable issues they had with MongoDB. https://news.ycombinator.com/item?id=16340462 https://news.ycombinator.com/item?id=16340462 I kind of assumed the fix is going to be a rewrite with Postgres or MySQL.

redwood 9y ago
Another way of reading that post is that MongoDB is being used for some of the highest throughput concurrent workloads out there... and those are always hard to optimize. Doing a lift and shift for a "grass is greener" alternate solution is not a clear cut path to victory at all... but it's certainly a giant science project to contemplate.
- stevenwoo 9y ago
  Yes, I kind of wish MongoDB had come out and said what they are doing to help Epic Games here if this is something to address that issue or what the plan/thoughts are on the most newsworthy usage of MongoDB in a while.
  - drmirror 9y ago
    I am one of the team of MongoDB engineers working with Epic on this issue, and I can assure you that the situation is under control and we have everything in place to scale this application to much higher numbers. However, we're not publishing details about our support cases, especially while they are in progress. That is something for Epic to decide, and I do assume they will eventually say in public just how well MongoDB is, in fact, performing for them.
    - stevenwoo 9y ago
      Thanks for responding. I look forward to Epic updating their post mortem.
  - threeseed 9y ago
    Not sure if you are aware but what you are asking for never happens. It is not professional or appropriate for vendors to be revealing (a) that clients are having issues and need support and (b) the specific workings of technologies or processes within the client's business.
munk-a 9y ago
Yes, come over to the actually greener lawn of Postgres, it is amazing how much more powerful than MySQL it has become with recursive CTEs https://www.postgresql.org/docs/current/static/queries-with.html https://www.postgresql.org/docs/current/static/queries-with.... and some rather amazing JSON support https://www.postgresql.org/docs/current/static/functions-aggregate.html https://www.postgresql.org/docs/current/static/functions-agg...
- derekperkins 9y ago
  JSON has been great in MySQL for a few years now since 5.7, and recursive CTEs are coming in the next couple months with 8.0. I don't think you can make a wrong choice between the two these days, but choosing Mongo over either is almost always the wrong decision. https://www.infoworld.com/article/3228154/sql/whats-new-in-mysql-80.html https://www.infoworld.com/article/3228154/sql/whats-new-in-m...
- takeda 9y ago
  This is actually what I think the biggest power of JSONB is. I can for example use jsonb_agg() and get a hierarchical response for 1:N joins. It returns JSON value even though neither of the columns contain JSON. Previously in that scenario I would either need to make more than one query or get a response that has a lot of data repeated.
hobofan 9y ago
> mentioned in passing they had stumped all the experts they could find to look at unsolveable issues they had with MongoDB That's not really what the article is saying, unless we interpret the following text differently. "We’ve flown Mongo experts on-site to analyze our DB and usage, as well as provide real-time support during heavy load on weekends." -> "We have started to look into the problem together with experts" and not "Experts have tried and failed".
Thaxll 9y ago
- That's not what they said. - You think people replace a MongoDB cluster by a single Posgres instance? You guys should really use HA, cluster in real life and stop reading reddit / HN and the hype behind PG, with 3.5M+ CCU no one would use an architecture with a single master / slave ( that's what pg is ). MongoDB / MySQL have bad press by people that never used it in real life and just repeat what they read online. I could tell you horror story about pg not have an official replication system until 2011 when pg 9.0 landed.
- woolvalley 9y ago
  You can cluster with postgres & mysql, but you have to implement the clustering / sharding logic yourself. Nowadays I would just use redis & cassandra if you need something beyond a collection of postgres instances. Most projects do not.
  - dankohn1 9y ago
    For clustering MySQL, take a look at Vitess. It was developed at YouTube and recently adopted into CNCF: https://www.cncf.io/blog/2018/02/05/cncf-host-vitess/ https://www.cncf.io/blog/2018/02/05/cncf-host-vitess/ (Disclosure: I'm executive director of CNCF.)
- stevenwoo 9y ago
  I feel I need to quote the post mortem back at you so you point out where I mis read the quote. "Our top focus right now is to ensure service availability. Our next steps are below: Identify and resolve the root cause of our DB performance issues. We’ve flown Mongo experts on-site to analyze our DB and usage, as well as provide real-time support during heavy load on weekends." How does that disagree with my post?
  - Thaxll 9y ago
    You're implying they couldn't fix MongoDB or reached the limit which is false. In the current ( HN ) post they said they fixed it, I'm pretty sure they didn't have any experience in DBs in the first place hence why they asked for help. Nowhere in the original post they mention issues related to MongoDB itself it was probably bad design on their side.
karmakaze 9y ago
Funny to list MySQL here which preceded MongoDB with cheap/fast now, correctness later.

redwood 9y ago

Another way of reading that post is that MongoDB is being used for some of the highest throughput concurrent workloads out there... and those are always hard to optimize. Doing a lift and shift for a "grass is greener" alternate solution is not a clear cut path to victory at all... but it's certainly a giant science project to contemplate.

stevenwoo 9y ago
Yes, I kind of wish MongoDB had come out and said what they are doing to help Epic Games here if this is something to address that issue or what the plan/thoughts are on the most newsworthy usage of MongoDB in a while.
- drmirror 9y ago
  I am one of the team of MongoDB engineers working with Epic on this issue, and I can assure you that the situation is under control and we have everything in place to scale this application to much higher numbers. However, we're not publishing details about our support cases, especially while they are in progress. That is something for Epic to decide, and I do assume they will eventually say in public just how well MongoDB is, in fact, performing for them.
  - stevenwoo 9y ago
    Thanks for responding. I look forward to Epic updating their post mortem.
- threeseed 9y ago
  Not sure if you are aware but what you are asking for never happens. It is not professional or appropriate for vendors to be revealing (a) that clients are having issues and need support and (b) the specific workings of technologies or processes within the client's business.

munk-a 9y ago

Yes, come over to the actually greener lawn of Postgres, it is amazing how much more powerful than MySQL it has become with recursive CTEs https://www.postgresql.org/docs/current/static/queries-with.html https://www.postgresql.org/docs/current/static/queries-with.... and some rather amazing JSON support https://www.postgresql.org/docs/current/static/functions-aggregate.html https://www.postgresql.org/docs/current/static/functions-agg...

derekperkins 9y ago
JSON has been great in MySQL for a few years now since 5.7, and recursive CTEs are coming in the next couple months with 8.0. I don't think you can make a wrong choice between the two these days, but choosing Mongo over either is almost always the wrong decision. https://www.infoworld.com/article/3228154/sql/whats-new-in-mysql-80.html https://www.infoworld.com/article/3228154/sql/whats-new-in-m...
takeda 9y ago
This is actually what I think the biggest power of JSONB is. I can for example use jsonb_agg() and get a hierarchical response for 1:N joins. It returns JSON value even though neither of the columns contain JSON. Previously in that scenario I would either need to make more than one query or get a response that has a lot of data repeated.

hobofan 9y ago

> mentioned in passing they had stumped all the experts they could find to look at unsolveable issues they had with MongoDB That's not really what the article is saying, unless we interpret the following text differently. "We’ve flown Mongo experts on-site to analyze our DB and usage, as well as provide real-time support during heavy load on weekends." -> "We have started to look into the problem together with experts" and not "Experts have tried and failed".

Thaxll 9y ago

- That's not what they said. - You think people replace a MongoDB cluster by a single Posgres instance? You guys should really use HA, cluster in real life and stop reading reddit / HN and the hype behind PG, with 3.5M+ CCU no one would use an architecture with a single master / slave ( that's what pg is ). MongoDB / MySQL have bad press by people that never used it in real life and just repeat what they read online. I could tell you horror story about pg not have an official replication system until 2011 when pg 9.0 landed.

woolvalley 9y ago
You can cluster with postgres & mysql, but you have to implement the clustering / sharding logic yourself. Nowadays I would just use redis & cassandra if you need something beyond a collection of postgres instances. Most projects do not.
- dankohn1 9y ago
  For clustering MySQL, take a look at Vitess. It was developed at YouTube and recently adopted into CNCF: https://www.cncf.io/blog/2018/02/05/cncf-host-vitess/ https://www.cncf.io/blog/2018/02/05/cncf-host-vitess/ (Disclosure: I'm executive director of CNCF.)
stevenwoo 9y ago
I feel I need to quote the post mortem back at you so you point out where I mis read the quote. "Our top focus right now is to ensure service availability. Our next steps are below: Identify and resolve the root cause of our DB performance issues. We’ve flown Mongo experts on-site to analyze our DB and usage, as well as provide real-time support during heavy load on weekends." How does that disagree with my post?
- Thaxll 9y ago
  You're implying they couldn't fix MongoDB or reached the limit which is false. In the current ( HN ) post they said they fixed it, I'm pretty sure they didn't have any experience in DBs in the first place hence why they asked for help. Nowhere in the original post they mention issues related to MongoDB itself it was probably bad design on their side.

karmakaze 9y ago

Funny to list MySQL here which preceded MongoDB with cheap/fast now, correctness later.