4 ms·

i usually start projects with postgres this days. i have reached the tens of millions of rows threshold without breaking a sweat, but is there any good reason p

i usually start projects with postgres this days. i have reached the tens of millions of rows threshold without breaking a sweat, but is there any good reason postgres can't handle into the billions or trillions? any well known products at that scale that are known to use postgres?

mxscho 2y ago
Just the raw amount of data is not enough metrics to judge whether postgres is "enough". They seem to value horizontal scalability e.g. in terms of write throughput, which is easier to handle with something like their solution compared to postgres.
bastawhiz 2y ago
Postgres can pretty easily scale to billions or trillions of rows. It forces you to think carefully about how you query that data, though, and I think most beginners would find themselves in deep trouble jumping into the deep end.
- qntmfred 2y ago
  > most beginners would find themselves in deep trouble jumping into the deep end probably true for any database platform. postgres probably easier for beginners than cassandra
postgresfan 2y ago
The problem with Postgres is that you have to read the doc (boring), sometimes read database books beyond chapter 2 (lmao nerds). This filters out 99% of software "engineers". So it's better to use KookaburaDB, version 0.2 just got released. It's written in Rust, and it's modern of course (whatever that fucking means: config written in YAML I guess? Complicated build and deployment?)
- consteval 2y ago
  Thank you prostgresfan, it's nice to see a completely unbiased source on database technologies. Jokes aside, you're largely right. Postgres really does cover 99.9% of database usecases. But, I think Discord might still fall outside of that. The problem is that Discord's scale pretty much requires heavy sharding. While you can make this word with Postgres, you can tell it was never designed out of the gate for this. IMO, Discord isn't even taking it far enough. Working under the assumption every server is its own isolated pocket, I see no reason not to have 1 database (or database-like thing) per server. Then it's truly a distributed system, which matches Discord's business use cases. I often find matching business use cases to technology like this can greatly simply architecture and reduce friction.

mxscho 2y ago

Just the raw amount of data is not enough metrics to judge whether postgres is "enough". They seem to value horizontal scalability e.g. in terms of write throughput, which is easier to handle with something like their solution compared to postgres.

bastawhiz 2y ago

Postgres can pretty easily scale to billions or trillions of rows. It forces you to think carefully about how you query that data, though, and I think most beginners would find themselves in deep trouble jumping into the deep end.

qntmfred 2y ago
> most beginners would find themselves in deep trouble jumping into the deep end probably true for any database platform. postgres probably easier for beginners than cassandra

postgresfan 2y ago

The problem with Postgres is that you have to read the doc (boring), sometimes read database books beyond chapter 2 (lmao nerds). This filters out 99% of software "engineers". So it's better to use KookaburaDB, version 0.2 just got released. It's written in Rust, and it's modern of course (whatever that fucking means: config written in YAML I guess? Complicated build and deployment?)

consteval 2y ago
Thank you prostgresfan, it's nice to see a completely unbiased source on database technologies. Jokes aside, you're largely right. Postgres really does cover 99.9% of database usecases. But, I think Discord might still fall outside of that. The problem is that Discord's scale pretty much requires heavy sharding. While you can make this word with Postgres, you can tell it was never designed out of the gate for this. IMO, Discord isn't even taking it far enough. Working under the assumption every server is its own isolated pocket, I see no reason not to have 1 database (or database-like thing) per server. Then it's truly a distributed system, which matches Discord's business use cases. I often find matching business use cases to technology like this can greatly simply architecture and reduce friction.