4 ms·
> Do you really need a queue? (Alternative: periodic polling of a DB) In my experience it’s not the reads, but the writes that are hard to scale up. Reading is
by drdaeman 1y ago
> Do you really need a queue? (Alternative: periodic polling of a DB)
In my experience it’s not the reads, but the writes that are hard to scale up. Reading is cheap and can be sometimes done off a replica. Writing to a PostgreSQL at high sustained rate requires careful tuning and designs. A stream of UPDATEs can be very painful, INSERTs aren’t cheap, and even a batched COPY blocks can be tricky.
- bostik 1y agoPlus of course you can take out the primary even with a read from a replica. It's not a trivial feat, but you can achieve it with the combination of streaming replication and an hours-long read from the replica for massive analytical workloads. For large reads Postgres will create temporary tables as needed, and when those in the replica end up far enough, the cascading effect through replication backpressure will cause primary to block further writes from getting through... The scars from that kind of outage will never truly heal.
- baq 1y agoIME (...don't ask) it's easy enough if you forget to set idle in transaction timeout, though I haven't... tried... on replicas
- sgarland 1y agoPostgres’ need (modulo running it on ZFS) for full-page writes [0], coupled with devs’ apparent need to use UUIDv4 everywhere - along with over-indexing - is a recipe to drag writes down to the floor, yes. 0: https://www.enterprisedb.com/blog/impact-full-page-writes https://www.enterprisedb.com/blog/impact-full-page-writes
- mobilemidget 1y agoDid you try uuidv7 yet?
- enether 11mo agoThe key question here is what is "high sustained rate" in numbers?