3 ms·
I've always found having to deal with background workers a pain in the butt. Not only do you have to set up additional servers (queue and worker servers), but a
by iamwil 13y ago
I've always found having to deal with background workers a pain in the butt. Not only do you have to set up additional servers (queue and worker servers), but also you run into problems as a result of your own naivety--such as not using a connection pool for workers so the startup time is too slow, or using a library that doesn't handle timeouts. Also, I find that I have little visibility into the queuing throughput, or if I chain jobs for workers (result of one worker goes to another worker), I have no idea if the data made it all the way through.
Does anyone else find it a pain, or just me?
Also, has anyone ever tried workers as a service, like iron.io? Are there others, and was it worth it?
- wslh 13y agoCan you give more information about the context where you are using background workers and queues?
- joevandyk 13y agoAmazon Simple Workflow might help: http://aws.amazon.com/swf/ http://aws.amazon.com/swf/ Personally, I use queue_classic + postgres triggers to keep track of each of the job history. So I have a history of all the jobs, when they started, when they finished, the data in each job, etc. But yes, it's a pain. http://torquebox.org/ http://torquebox.org/ is supposed to help with the pain of managing background jobs (and lots of other things) for Rails applications.