3 ms·
SKIP LOCKED is amazing for this kind of thing. I used it to build a transaction outbox a few years ago. One thing worth looking into if you do this in producti
by sarchertech 1y ago
SKIP LOCKED is amazing for this kind of thing. I used it to build a transaction outbox a few years ago.
One thing worth looking into if you do this in production is adding a way to add partitions such that each partition is single threaded. It’s the only way to guarantee ordering if your jobs are doing anything non-deterministic.
- boruto 1y agoWe have a system where each pod spins up around 30 scheduled job instances of one job each processing a "partition", then transaction outbox is queried with hash of identifier equating it to partition. We increased partition counts on sale days and it works well for us. Couple of gotchas we had were. 1) Using hashtext from postgres is sketchy. 2) Increasing partiton count is an orchestra which requires stopping the partition.