3 ms·
> scales a very long way This ignores the fine print. It scales a very long way under specific circumstances with specific workloads. The more write-heavy your
by tomnipotent 2mo ago
> scales a very long way
This ignores the fine print. It scales a very long way under specific circumstances with specific workloads. The more write-heavy your workload the less eloquently Postgres scales.
For OpenAI's use case you could swap Postgres with MySQL and it would scale just as well.
- ballon_monkey 2mo agoIf your DB is write heavy, tune it for writes...
- tomnipotent 2mo agoYou can't tune your way out of this constraint. The heap table and MVCC implementation put a hard ceiling on what can be accomplished without reaching for another tool.
- dewey 2mo agoWe are talking about Postgres as a queue here, which is heavy on tiny writes, reads and churning tables. The point is just that a RDMS like Postgres scales very far as a queue, it’s not a fight if Postgres or MySQL. Nobody is arguing for using it for everything and forever but for most company sizes it’s perfectly fine to not reach for a dedicated queuing tool if you already have PG running.
- tomnipotent 2mo ago> on tiny writes Postgres doesn't do "tiny writes" - it writes whole pages multiple times on every update even if you're only changing 4 bytes, combined with even more writes later on when vacuuming tables. This is one of the reasons why the historical advice was to not build high-volume queues on top of Postgres and why "oh look another post about queues on Postgres" keeps soliciting comments like this.