3 ms·
> And we guarantee that jobs won’t be picked up by more than one worker through simple read/write row-level locks. The new system is actually kind of absurdly s
by avinassh 4y ago
> And we guarantee that jobs won’t be picked up by more than one worker through simple read/write row-level locks. The new system is actually kind of absurdly simple when you look at it. And that’s a good thing. It’s also behaved flawlessly so far.
Wouldn't this lead to contention issue when a lot of multiple workers are involved?
- prpl 4y agoYou can do an a select statement skipping locked rows in postgres.
- zinclozenge 4y agoSpecifically you also want SELECT ... FOR UPDATE SKIP LOCKED;
- CBLT 4y agoI believe that problem is avoided by using SKIP LOCKED[0]. [0] https://www.2ndquadrant.com/en/blog/what-is-select-skip-locked-for-in-postgresql-9-5/ https://www.2ndquadrant.com/en/blog/what-is-select-skip-lock...
- fabian2k 4y agoMy understanding is that a naive implementation essentially serializes access to the queue table. So it works, but no matter how many requests you make in parallel, only one will be served at a time (unless you have a random component in the query). With SKIP LOCKED you can resolve this easily, as long as you know about the feature. But almost every tutorial and description of implementing a job queue in Postgres mentions this now.