4 ms·
> To make all of this run smoothly, we enqueue and dequeue thousands of jobs every day. The scale isn't large enough for this to at all be a worry. The biggest
by throwaway5959 4y ago
> To make all of this run smoothly, we enqueue and dequeue thousands of jobs every day.
The scale isn't large enough for this to at all be a worry. The biggest worry here I imagine is ensuring that a job isn't processed by multiple workers, which they solve with features built into Postgres.
Usually I caution against using a database as a queue, but in this case it removes a piece of the architecture that they have to manage and they're clearly more comfortable with SQL than RabbitMQ so it sounds like a good call.
- KrugerDunnings 4y agoIt is easy to avoid multiple workers processing the same task: `delete from task where id = (select id from task for update skip locked limit 1) returning *;`
- throwaway5959 4y agoI didn't say it was difficult, I just said it was the biggest concern. That looks correct.