5 ms·

I see the “Postgres is technology superior” take on hn all the time. Can you actually back that up? Can you explain why the dozens or so Postgres scale up solu

I see the “Postgres is technology superior” take on hn all the time. Can you actually back that up? Can you explain why the dozens or so Postgres scale up solutions don’t use real Postgres? Or why anyone at scale with Postgres migrates away? https://www.uber.com/en-US/blog/postgres-to-mysql-migration/ https://www.uber.com/en-US/blog/postgres-to-mysql-migration/ This outage would have been a lot less likely with MySQL because of redo logs which MySQL has had for a decade. But Postgres replication is under developed. https://about.gitlab.com/blog/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab... Facebook has evaluated every database on the planet and still uses MySQL. https://engineering.fb.com/2021/07/22/core-infra/mysql/ https://engineering.fb.com/2021/07/22/core-infra/mysql/ When MySQL has more users, can scale significantly more, runs some of the largest sites on earth, and even lends its storage engine to other databases like Dynamo you should backup your claims that Postgres is technically superior. Please.

otabdeveloper4 3y ago
Yes, Postgres is a toy database that can't do replication correctly. You are not wrong.
- keep_reading 3y ago
  Postgres replication features have lagged behind but were never dangerous. MySQL has multiple edge cases where things on the master will not happen on the slave depending on which replication method you've selected and which features you're using. MySQL is the issue here.
bbatha 3y ago
Postgres still has a ways to go for scale but has improved substantially in replication options since that 2017 article. Sharding has also seen a lot of upstream patches from enterprise db and other commercial providers since then. Upgrades in particular are still a sore spot. Also note that folks aren’t running giant vanilla MySQL clusters. They’re running it through vitess or home grown tools with similar functionality. Finally MySQL gets performance in these large setups by turning off features like foreign keys, acid guarantees etc. it’s awesome and powerful that you can do this in MySQL but it’s not apples to apples here. MySQL has also improved a lot on these dimensions in the last few years. The other reality here is that on nvme hardware with a few tb of ram you can get away with a basic reader writer replica setup for a long time.
- samlambert 3y ago
  Vanilla MySQL vs vanilla Postgres. MySQL still wins on scale. You run single MySQL servers with 70,000 connections, you can't get close on Postgres because of a fundamentally broken connection-per-process model. The Neon team suggested changing that and got flamed by the community. I was at GitHub in 2017 when that GitLab outage happened. We had 60 million users running on vanilla MySQL no Vitess at that point. We had 10 million users literally running on 3 MySQL servers. Don't pretend that even now Postgres can do that.
  - bbatha 3y ago
    Agreed However, you can absolutely run that many users on a Postgres cluster of that size today on modern hardware. Connections are absolutely a sore point and probably my number one pain point as a Postgres admin day to day. Pg bouncer is incredibly easy to run though. You rarely actually need 70,000 connections most will be idle most of the time anyway it’s not like MySQL can actually serve 70,000 queries on 70,000 connections simultaneously. But it certainly makes administration and the programming model more difficult.
    - samlambert 3y ago
      70k QPS is easy on a single MySQL.
      - bbatha 3y ago
        Same with Postgres. Qps != active connections.
    - iamdanieljohns 3y ago
      What do you think of Supavisor[0] ? [0] https://supabase.com/blog/supavisor-1-million https://supabase.com/blog/supavisor-1-million
  - justinclift 3y ago
    While MySQL being able to scale is nice and all, it's had serious problems with even keeping basic data integrity for the data people hand it... for decades.
jitl 3y ago
Postgres has a reputation for correctness and does have some more specialized features, column, and index types. It’s also totally fine and dandy at small-medium size. Fully transactional DDL and online schema migrations put it ahead of MySQL at those scales too. But I agree that MySQL is easier to use at high scale for its superior replication story, 3rd party tooling, and pragmatism. If you’ve only run small Postgres you don’t know the fear of the query planner suddenly going AWOL at 3am, or needing to shard your database under duress as you watch the Postgres transaction ID stick closer and closer to overflow because VACUUM won’t finish. You haven’t needed to stress out about the super low practical connection limit and build a cluster of Postgres proxies using PGBouncer or similar. Related blog post by a teammate: https://www.notion.so/blog/sharding-postgres-at-notion https://www.notion.so/blog/sharding-postgres-at-notion
v3ss0n 3y ago
MySQL have unfixable legacy code for scalability that world top database engineer gave up and asked users to stop using.. https://www.google.com/amp/s/www.theregister.com/AMP/2021/12/06/mysql_a_pretty_poor_database/ https://www.google.com/amp/s/www.theregister.com/AMP/2021/12... facebook was build using pho mysql since start, by the time they have billion dollar valuation 8ts too late for them to change.
- endisneigh 3y ago
  Considering the source take it with a grain of salt
  - v3ss0n 3y ago
    Oracle engineer who is tasked to work on MySQL optimizations is not a good source?
    - endisneigh 3y ago
      A single departing employees opinion is not definitive, no. Considering both YouTube and Facebook use modified MySQL I’m skeptical there’s some inherent structural disadvantages. Not to mention no technical reasoning is given at all, so yes take the article with a grain of salt.
- samlambert 3y ago
  They have reevaluated multiple times.

otabdeveloper4 3y ago

Yes, Postgres is a toy database that can't do replication correctly. You are not wrong.

keep_reading 3y ago
Postgres replication features have lagged behind but were never dangerous. MySQL has multiple edge cases where things on the master will not happen on the slave depending on which replication method you've selected and which features you're using. MySQL is the issue here.

bbatha 3y ago

Postgres still has a ways to go for scale but has improved substantially in replication options since that 2017 article. Sharding has also seen a lot of upstream patches from enterprise db and other commercial providers since then. Upgrades in particular are still a sore spot. Also note that folks aren’t running giant vanilla MySQL clusters. They’re running it through vitess or home grown tools with similar functionality. Finally MySQL gets performance in these large setups by turning off features like foreign keys, acid guarantees etc. it’s awesome and powerful that you can do this in MySQL but it’s not apples to apples here. MySQL has also improved a lot on these dimensions in the last few years. The other reality here is that on nvme hardware with a few tb of ram you can get away with a basic reader writer replica setup for a long time.

samlambert 3y ago
Vanilla MySQL vs vanilla Postgres. MySQL still wins on scale. You run single MySQL servers with 70,000 connections, you can't get close on Postgres because of a fundamentally broken connection-per-process model. The Neon team suggested changing that and got flamed by the community. I was at GitHub in 2017 when that GitLab outage happened. We had 60 million users running on vanilla MySQL no Vitess at that point. We had 10 million users literally running on 3 MySQL servers. Don't pretend that even now Postgres can do that.
- bbatha 3y ago
  Agreed However, you can absolutely run that many users on a Postgres cluster of that size today on modern hardware. Connections are absolutely a sore point and probably my number one pain point as a Postgres admin day to day. Pg bouncer is incredibly easy to run though. You rarely actually need 70,000 connections most will be idle most of the time anyway it’s not like MySQL can actually serve 70,000 queries on 70,000 connections simultaneously. But it certainly makes administration and the programming model more difficult.
  - samlambert 3y ago
    70k QPS is easy on a single MySQL.
    - bbatha 3y ago
      Same with Postgres. Qps != active connections.
  - iamdanieljohns 3y ago
    What do you think of Supavisor[0] ? [0] https://supabase.com/blog/supavisor-1-million https://supabase.com/blog/supavisor-1-million
- justinclift 3y ago
  While MySQL being able to scale is nice and all, it's had serious problems with even keeping basic data integrity for the data people hand it... for decades.

jitl 3y ago

Postgres has a reputation for correctness and does have some more specialized features, column, and index types. It’s also totally fine and dandy at small-medium size. Fully transactional DDL and online schema migrations put it ahead of MySQL at those scales too. But I agree that MySQL is easier to use at high scale for its superior replication story, 3rd party tooling, and pragmatism. If you’ve only run small Postgres you don’t know the fear of the query planner suddenly going AWOL at 3am, or needing to shard your database under duress as you watch the Postgres transaction ID stick closer and closer to overflow because VACUUM won’t finish. You haven’t needed to stress out about the super low practical connection limit and build a cluster of Postgres proxies using PGBouncer or similar. Related blog post by a teammate: https://www.notion.so/blog/sharding-postgres-at-notion https://www.notion.so/blog/sharding-postgres-at-notion

v3ss0n 3y ago

MySQL have unfixable legacy code for scalability that world top database engineer gave up and asked users to stop using.. https://www.google.com/amp/s/www.theregister.com/AMP/2021/12/06/mysql_a_pretty_poor_database/ https://www.google.com/amp/s/www.theregister.com/AMP/2021/12... facebook was build using pho mysql since start, by the time they have billion dollar valuation 8ts too late for them to change.

endisneigh 3y ago
Considering the source take it with a grain of salt
- v3ss0n 3y ago
  Oracle engineer who is tasked to work on MySQL optimizations is not a good source?
  - endisneigh 3y ago
    A single departing employees opinion is not definitive, no. Considering both YouTube and Facebook use modified MySQL I’m skeptical there’s some inherent structural disadvantages. Not to mention no technical reasoning is given at all, so yes take the article with a grain of salt.
samlambert 3y ago
They have reevaluated multiple times.