In our setup (using Sql Server, alas, not redis but same concept applies) the worker process sets the lock and the time of acquiring it. Other workers look for

In our setup (using Sql Server, alas, not redis but same concept applies) the worker process sets the lock and the time of acquiring it. Other workers look for pending jobs that either don't have the lock set or the lock has expired where the expiry time is 5min + lock time. Since each 'job' is a small amount of work, 5 minutes is fairly generous. If a worker goes down because of a system crash or network failure, then that job wouldn't get processed again for 5 minutes. That's (for us) an acceptable overhead. There's a chance the job gets processed twice, if it gets locked, work gets done, but releasing the lock fails. That's a trade-off we're willing to make as we can't really think of any better solution. I believe, since redis has built-in support for expiring records, this maybe even easier. To process job 332, you would try to check the existence of 'lock:332' and then set it if it didn't exist, and give it an expiry time. Would be interested in hearing how other people solve these types of issues.