3 ms·
The pattern I follow is that workers immediately poll once when they start - this means that when your workers restart after crashing, they'll pick up any missi
by colinchartier 7y ago
The pattern I follow is that workers immediately poll once when they start - this means that when your workers restart after crashing, they'll pick up any missing jobs.
- pm90 7y agoI'm not seeing how this recovers from the failure scenario where a worker grabbed an event and then immediately crashed.
- tjungblut 7y agoYou need some timeout and/or leasing mechanism to make it work in this scenario.
- Rapzid 7y agoThe comment you are replying to wasn't addressing that scenario. Typically you would implement visibility timeouts and other such stuff. Depending on the use case you could make specific optimizations or keep it generic and have SQS like semantics or something.
- disiplus 7y agoat this point why not use something like rabbitmq and not reinvent it in postgres.
- rapind 7y agoBecause then you're running and maintaining another service. Might make sense, but depends.
- stickfigure 7y agoBecause it's trivially easy. The dequeue operation updates the last_lease time; the query makes sure you don't dequeue something that was recently leased. It's far, far easier than setting up a separate job queue, especially if you are already using an RDBMS.