4 ms·
Interestingly there is a Sidekiq compatible job queue written in Go that has a similar name: go-workers https://github.com/jrallison/go-workers https://github.c
by jemeshsu 13y ago
Interestingly there is a Sidekiq compatible job queue written in Go that has a similar name: go-workers https://github.com/jrallison/go-workers https://github.com/jrallison/go-workers
Wondering how these ruby/rails inspired work queues compare with NSQ.
- jonnii 13y agoI'm using this and it works great. We even managed to get it deploy to heroku, which was pretty magic.
- benmanns 13y agoHow do you handle when Heroku sends a SIGTERM to kill your process? I couldn't find a way to preempt running workers, so everything running on Heroku has to finish within 10 seconds or you can lose it.
- jonnii 13y agoI'm not sure tbh, we haven't had that problem yet.
- jrallison 13y agoHey Ben, Author of go-workers here. On SIGTERM, I stop accepting new work, and wait for all running workers to finish before halting. If workers take longer than the 10 seconds Heroku gives you, go-workers uses reliable queueing (using http://redis.io/commands/brpoplpush http://redis.io/commands/brpoplpush) so the job will run again next time you start up the process.
- benmanns 13y agoOkay, cool. I do the same thing on the polling side, but don't use reliable queueing yet. I think that is probably the best way to handle the failures.
- rargulati 13y agoWe're in the midst of switching from quite a large sidekiq deployment to inbound work being done in NSQ. At a high level, NSQ allows for different strategies like fanout (having same msg processed by different kinds of workers) without having to build them into a lib. This involves outgoing API requests and minimal scraping. We're keeping our outbound work in sidekiq for now (less throughput; not as necessary to have fast deserialization). It would be interesting to move all the workers to Go (and explore the advantages of the libraries linked here) as well.