3 ms·
"two messages with quick ack, one message at start, one message at end from worker." how do you deal with worker getting killed in that scenario ? if you can't
by bsaul 4y ago
"two messages with quick ack, one message at start, one message at end from worker."
how do you deal with worker getting killed in that scenario ? if you can't rely on the queue job state then you need another whole set of code somewhere to handle timeouts & retries, don't you ?
- tuyiown 4y agoFor long async processes: A worker, or at least, the consumer receiving the messages and spawning sub workers, when getting killed, should restart at some point, inspecting jobs ids unfinished, the one stored just before ack'ing reception, and either resume work or notify failure. Answering is the worker sole responsibility, and no viable implementation of an app can reliably try to substitute to that. As for timeouts, they are necessary when waiting for answers from services that are not under your control, eg, you cannot assume they will answer you in all cases. Under your control, you have to keep yourself in a situation where worker response is guaranteed, success or failure.
- bsaul 4y agoI feel like this would reimplement a lot of the machinery that's built-in the job queue (atomic state locking to take ownership of the job, with deadlocking prevention mechanism to make sure a killed process doesn't keep that job locked, and timeouts). At this point, why not throw the whole job queue away and simply use that system ?