6 ms·

Postgres still has a ways to go for scale but has improved substantially in replication options since that 2017 article. Sharding has also seen a lot of upstrea

by bbatha 3y ago

Postgres still has a ways to go for scale but has improved substantially in replication options since that 2017 article. Sharding has also seen a lot of upstream patches from enterprise db and other commercial providers since then. Upgrades in particular are still a sore spot. Also note that folks aren’t running giant vanilla MySQL clusters. They’re running it through vitess or home grown tools with similar functionality. Finally MySQL gets performance in these large setups by turning off features like foreign keys, acid guarantees etc. it’s awesome and powerful that you can do this in MySQL but it’s not apples to apples here. MySQL has also improved a lot on these dimensions in the last few years. The other reality here is that on nvme hardware with a few tb of ram you can get away with a basic reader writer replica setup for a long time.

samlambert 3y ago
Vanilla MySQL vs vanilla Postgres. MySQL still wins on scale. You run single MySQL servers with 70,000 connections, you can't get close on Postgres because of a fundamentally broken connection-per-process model. The Neon team suggested changing that and got flamed by the community. I was at GitHub in 2017 when that GitLab outage happened. We had 60 million users running on vanilla MySQL no Vitess at that point. We had 10 million users literally running on 3 MySQL servers. Don't pretend that even now Postgres can do that.
- bbatha 3y ago
  Agreed However, you can absolutely run that many users on a Postgres cluster of that size today on modern hardware. Connections are absolutely a sore point and probably my number one pain point as a Postgres admin day to day. Pg bouncer is incredibly easy to run though. You rarely actually need 70,000 connections most will be idle most of the time anyway it’s not like MySQL can actually serve 70,000 queries on 70,000 connections simultaneously. But it certainly makes administration and the programming model more difficult.
  - samlambert 3y ago
    70k QPS is easy on a single MySQL.
    - bbatha 3y ago
      Same with Postgres. Qps != active connections.
  - iamdanieljohns 3y ago
    What do you think of Supavisor[0] ? [0] https://supabase.com/blog/supavisor-1-million https://supabase.com/blog/supavisor-1-million
- justinclift 3y ago
  While MySQL being able to scale is nice and all, it's had serious problems with even keeping basic data integrity for the data people hand it... for decades.
  - bombcar 3y ago
    This is the root of it all - for years and years MySQL was perfectly capable of doing the needful, but it shipped with default configurations that would do things like use "not_quite_really_UTF8" and "myisam_tables_please_no_acid". If you knew enough to switch to innodb and "mysql_real_utf8_string" or whatever it was, you had a quite performant and stable system.
    - justinclift 3y ago
      There have also been numerous horror stories where people discovered (after the fact) that MySQL wasn't storing their data as given, but instead internally silently converting it to some other format/type and storing that. People that directly experienced this very thing will probably chime in to give examples (etc). :)
      - jarym 3y ago
        Yep - ME! Happened to ME! Now some will blame bad development, should have caught it in the application code via testing, etc. A 'true' DBMS will provide some guarantees and rather fail a transaction than do a 'best endeavours' job with it.
    - anonacct37 3y ago
      You hit the nail on the head, those were my top two complaints, but the rabbit hole goes deep. It's not an exaggeration to say that trying to keep mysql from corrupting your data feels like a full-time job. Mysql in many ways, especially as used by massive social media companies early on was closer to a SQLish DSL for a non-transactional kv store than a "real" ACID database. That's why "but it can do 70k rps" is just noise to someone. I don't care how fast it can lose my data. I'm all for people needing to read the docs but man a database burns trust pretty fast when it's so easy to have it pretend to support transactions or pretend to support utf8.
- CAP_NET_ADMIN 3y ago
  I have 5 million users running on PostgreSQL, 3 dedicated servers (primary + 2 replicas), we use less than 190 connections, vanilla PostgreSQL and no things like pgbouncer.
  - samlambert 3y ago
    Right. In 2023. This was possible in 2013 with MySQL.
    - winrid 3y ago
      It was possible with PG in 2013 too, with only 190 connections...
samlambert 3y ago
Also Vitess has foreign keys shipping every soon.
evanelias 3y ago
> Finally MySQL gets performance in these large setups by turning off features like foreign keys, acid guarantees etc. Large MySQL users are generally not turning off acid guarantees. That is nonsense. What settings are you even specifically referring to with this? On the durability side, large MySQL shops only mess with innodb_flush_log_at_trx_commit if there's a safe distributed durability mechanism in use (e.g. synchronous or semi-synchronous replication). Or perhaps if the data is ephemeral/recomputable, but that's a similar use-case to running Postgres on a ramdisk. On the isolation side, MySQL supports READ UNCOMMITTED (unlike Postgres) but it is very rarely used. Only useful for some specific situations. fwiw MS SQL Server supports it too, it's not a MySQL specific thing.

by bbatha 3y ago

samlambert 3y ago

Vanilla MySQL vs vanilla Postgres. MySQL still wins on scale. You run single MySQL servers with 70,000 connections, you can't get close on Postgres because of a fundamentally broken connection-per-process model. The Neon team suggested changing that and got flamed by the community. I was at GitHub in 2017 when that GitLab outage happened. We had 60 million users running on vanilla MySQL no Vitess at that point. We had 10 million users literally running on 3 MySQL servers. Don't pretend that even now Postgres can do that.

bbatha 3y ago
Agreed However, you can absolutely run that many users on a Postgres cluster of that size today on modern hardware. Connections are absolutely a sore point and probably my number one pain point as a Postgres admin day to day. Pg bouncer is incredibly easy to run though. You rarely actually need 70,000 connections most will be idle most of the time anyway it’s not like MySQL can actually serve 70,000 queries on 70,000 connections simultaneously. But it certainly makes administration and the programming model more difficult.
- samlambert 3y ago
  70k QPS is easy on a single MySQL.
  - bbatha 3y ago
    Same with Postgres. Qps != active connections.
- iamdanieljohns 3y ago
  What do you think of Supavisor[0] ? [0] https://supabase.com/blog/supavisor-1-million https://supabase.com/blog/supavisor-1-million
justinclift 3y ago
While MySQL being able to scale is nice and all, it's had serious problems with even keeping basic data integrity for the data people hand it... for decades.
- bombcar 3y ago
  This is the root of it all - for years and years MySQL was perfectly capable of doing the needful, but it shipped with default configurations that would do things like use "not_quite_really_UTF8" and "myisam_tables_please_no_acid". If you knew enough to switch to innodb and "mysql_real_utf8_string" or whatever it was, you had a quite performant and stable system.
  - justinclift 3y ago
    There have also been numerous horror stories where people discovered (after the fact) that MySQL wasn't storing their data as given, but instead internally silently converting it to some other format/type and storing that. People that directly experienced this very thing will probably chime in to give examples (etc). :)
    - jarym 3y ago
      Yep - ME! Happened to ME! Now some will blame bad development, should have caught it in the application code via testing, etc. A 'true' DBMS will provide some guarantees and rather fail a transaction than do a 'best endeavours' job with it.
  - anonacct37 3y ago
    You hit the nail on the head, those were my top two complaints, but the rabbit hole goes deep. It's not an exaggeration to say that trying to keep mysql from corrupting your data feels like a full-time job. Mysql in many ways, especially as used by massive social media companies early on was closer to a SQLish DSL for a non-transactional kv store than a "real" ACID database. That's why "but it can do 70k rps" is just noise to someone. I don't care how fast it can lose my data. I'm all for people needing to read the docs but man a database burns trust pretty fast when it's so easy to have it pretend to support transactions or pretend to support utf8.
CAP_NET_ADMIN 3y ago
I have 5 million users running on PostgreSQL, 3 dedicated servers (primary + 2 replicas), we use less than 190 connections, vanilla PostgreSQL and no things like pgbouncer.
- samlambert 3y ago
  Right. In 2023. This was possible in 2013 with MySQL.
  - winrid 3y ago
    It was possible with PG in 2013 too, with only 190 connections...

samlambert 3y ago

Also Vitess has foreign keys shipping every soon.

evanelias 3y ago

> Finally MySQL gets performance in these large setups by turning off features like foreign keys, acid guarantees etc. Large MySQL users are generally not turning off acid guarantees. That is nonsense. What settings are you even specifically referring to with this? On the durability side, large MySQL shops only mess with innodb_flush_log_at_trx_commit if there's a safe distributed durability mechanism in use (e.g. synchronous or semi-synchronous replication). Or perhaps if the data is ephemeral/recomputable, but that's a similar use-case to running Postgres on a ramdisk. On the isolation side, MySQL supports READ UNCOMMITTED (unlike Postgres) but it is very rarely used. Only useful for some specific situations. fwiw MS SQL Server supports it too, it's not a MySQL specific thing.