5 ms·

ActiveJob – gem for defining background jobs in Rails

64 points by Gonzih 12y ago

callumjones 12y ago
I like idea of standardising a way for jobs to be defined and then shipped off to a background server. The adapter registration looks neat. But by the looks of it the library bundles in the adapters to talk directly to these worker frameworks (currently it only is Resque). Are we going to see an active merging in of adapters or will at some point will the response be 'we won't be allowing any more adapters to bundled'.
- byroot 12y ago
  You don't really need the adapters to be bundled. The queue libraries can easily bundle their adapters in their gem or in 3rd party gems like sidekiq-rails etc
  - callumjones 12y ago
    Totally, I see that in the class variable. I'm just curious when this library will blow out.
jayroh 12y ago
I ran into this a few days ago when I was searching for a common background job / queue interface: https://github.com/dwbutler/multi_worker https://github.com/dwbutler/multi_worker Haven't tried it yet but looks promising.
igravious 12y ago
How does this differ from good ol' cron and friends? Could someone tell me?
- byroot 12y ago
  It just has nothing to do with cron. Cron is a POSIX task scheduler, this is a Ruby normalized API for background job systems.
- patio11 12y ago
  Generally one runs defined cron jobs on a schedule set in advance. This allows you to queue up arbitrary methods to fire at arbitrary times, generally "a short while after this HTTP request has been dealt with." This is an extraordinarily common pattern in web applications. My Rails apps do it hundreds of thousands of times per day, often to avoid hitting an external API during a request/response cycle. (I use Delayed::Job. This would not be my #1 recommendation for this pattern in 2014. Take a look at Sidekiq, Resque, or this new thing instead.)
  - jhuckestein 12y ago
    I'm pretty sure ActiveJob is just a standardized interface to delayed job, resque, sidekiq etc. If you ever wanted to upgrade form delayed job, this could come in handy for you.
  - igravious 12y ago
    I see, cron is absolute time events (every 5 minutes say or 2 minutes past the hour or once a week) and Active Job standardizes an interface to queue events relatively (5 minutes from this HTTP request, ten minutes from now). Is there a way cron could be extended to handle the relative case?
    - tomblomfield 12y ago
      The main value of these systems is normally not "relative queuing" (although that is sometimes done). Often, there are certain activities you might want to do in a way that doesn't block the request/response cycle, but still have them performed as soon as possible. For example, a user buys a widget on your site - send a receipt email. You don't want the "Thanks for buying" success page response to wait for your servers to compose & send the email, so you offload the email to your queue system.
- djkz 12y ago
  To give an example let's say you have a mailing list with 1000 people and you want to send out notification. You push "notify " button, but you don't want to wait for your app to go through 1000 people and send them email until it responds again. You also don't want to wait until midnight for crontab to start processing all requests. Background jobs takes that 1000 deliveries job and puts it into the background queue, then a worker starts sending those emails right away. Meanwhile your app returns from your request instantly so you can continue doing what you were doing while the job is executed in the background.
- netghost 12y ago
  Just to pile on to the answers. Cron is for scheduling work, delayed jobs are for out of band work. For instance: * Use cron to dump a database backup nightly * Use some job system to send a user an email after they sign up There's a lot of overlap, and if used cleverly (which may not be smart), they can both achieve the same goals.
- rmchugh 12y ago
  Another thing to bear in mind is that cron won't have access to your Rails environment per default. If for example you used cron to call a rake script which needed the Rails environment, you would be loading the Rails env every time, which is pretty heavy.
rubiquity 12y ago
Rails 4.x was originally supposed to come with a Queueing API, but it was delayed/removed for reasons. It's nice to see this being picked back up and implemented. There's a ton of fragmentation out there in terms of background job libraries, there's also a lot of apps still using DelayedJob and others that "can't" upgrade to something better such as Sidekiq due to the sheer amount of work required. Standardizing on an API for this will help a lot and enable easily moving to the Latest and Greatest™ queueing library.
kalleth 12y ago
Sigh. This is like that timecop thing -- dumping more stuff in core that lives better as part of a gem. Resque, DelayedJob, Sidekiq... All have their drawbacks/benefits. I know this can just work as a 'wrapper', but in an agency environment, i've used queueing on about 20% of sites. Absolutely no reason to want to put this in core imo.
- unfunco 12y ago
  It being included will have no performance overhead since it's only loaded if it's used. I'm personally for this idea, I've used DJ, Resque and Sidekiq, all in the same project (increasing load), and in that order; It would have been much nicer to have to only change the Gemfile when I made the switch. Your comment, whilst a perfectly valid concern, reminds me of an article DHH posted: http://david.heinemeierhansson.com/2012/rails-is-omakase.html http://david.heinemeierhansson.com/2012/rails-is-omakase.htm...
  - kalleth 12y ago
    Performance wasn't my concern at all. Overcomplicating the rails 'core' is my concern. I'd much rather start from a lightweight 'base' which solves most of the common cases and pull in extras as required for what i'm working on. Rails should be a framework that allows you to layer extra functionality ontop of it.
    - _pius 12y ago
      Overcomplicating the rails 'core' is my concern. I'd much rather start from a lightweight 'base' which solves most of the common cases ... Background jobs are a common case. How many web applications never send an e-mail?
      - mackwic 12y ago
        He probably sends emails synchronously.
- amorphid 12y ago
dpeck 12y ago
I don't really see the need for this. A unified interface could be nice, but this will also limit the scope that background processors will operate in. Besides that the amount of rails projects that need background jobs aren't that large of a percentage comparatively. I'd rather see some more interesting things going into core, such as https://github.com/kickstarter/rack-attack https://github.com/kickstarter/rack-attack, that should be used by nearly every app.
timtas 12y ago
Having recently migrated from Resque to Sidekiq, I can attest that this solves a problem, albeit not terribly a big one. As with any abstraction, implementation-specific features make leaks. So far, the README doesn't say how I will control features like retries and queue weights, but maybe that's coming soon.

64 points by Gonzih 12y ago

I like idea of standardising a way for jobs to be defined and then shipped off to a background server. The adapter registration looks neat. But by the looks of it the library bundles in the adapters to talk directly to these worker frameworks (currently it only is Resque). Are we going to see an active merging in of adapters or will at some point will the response be 'we won't be allowing any more adapters to bundled'.

byroot 12y ago
You don't really need the adapters to be bundled. The queue libraries can easily bundle their adapters in their gem or in 3rd party gems like sidekiq-rails etc
- callumjones 12y ago
  Totally, I see that in the class variable. I'm just curious when this library will blow out.

jayroh 12y ago

I ran into this a few days ago when I was searching for a common background job / queue interface: https://github.com/dwbutler/multi_worker https://github.com/dwbutler/multi_worker Haven't tried it yet but looks promising.

igravious 12y ago

How does this differ from good ol' cron and friends? Could someone tell me?

byroot 12y ago
It just has nothing to do with cron. Cron is a POSIX task scheduler, this is a Ruby normalized API for background job systems.
patio11 12y ago
Generally one runs defined cron jobs on a schedule set in advance. This allows you to queue up arbitrary methods to fire at arbitrary times, generally "a short while after this HTTP request has been dealt with." This is an extraordinarily common pattern in web applications. My Rails apps do it hundreds of thousands of times per day, often to avoid hitting an external API during a request/response cycle. (I use Delayed::Job. This would not be my #1 recommendation for this pattern in 2014. Take a look at Sidekiq, Resque, or this new thing instead.)
- jhuckestein 12y ago
  I'm pretty sure ActiveJob is just a standardized interface to delayed job, resque, sidekiq etc. If you ever wanted to upgrade form delayed job, this could come in handy for you.
- igravious 12y ago
  I see, cron is absolute time events (every 5 minutes say or 2 minutes past the hour or once a week) and Active Job standardizes an interface to queue events relatively (5 minutes from this HTTP request, ten minutes from now). Is there a way cron could be extended to handle the relative case?
  - tomblomfield 12y ago
    The main value of these systems is normally not "relative queuing" (although that is sometimes done). Often, there are certain activities you might want to do in a way that doesn't block the request/response cycle, but still have them performed as soon as possible. For example, a user buys a widget on your site - send a receipt email. You don't want the "Thanks for buying" success page response to wait for your servers to compose & send the email, so you offload the email to your queue system.
djkz 12y ago
To give an example let's say you have a mailing list with 1000 people and you want to send out notification. You push "notify " button, but you don't want to wait for your app to go through 1000 people and send them email until it responds again. You also don't want to wait until midnight for crontab to start processing all requests. Background jobs takes that 1000 deliveries job and puts it into the background queue, then a worker starts sending those emails right away. Meanwhile your app returns from your request instantly so you can continue doing what you were doing while the job is executed in the background.
netghost 12y ago
Just to pile on to the answers. Cron is for scheduling work, delayed jobs are for out of band work. For instance: * Use cron to dump a database backup nightly * Use some job system to send a user an email after they sign up There's a lot of overlap, and if used cleverly (which may not be smart), they can both achieve the same goals.
rmchugh 12y ago
Another thing to bear in mind is that cron won't have access to your Rails environment per default. If for example you used cron to call a rake script which needed the Rails environment, you would be loading the Rails env every time, which is pretty heavy.

rubiquity 12y ago

Rails 4.x was originally supposed to come with a Queueing API, but it was delayed/removed for reasons. It's nice to see this being picked back up and implemented. There's a ton of fragmentation out there in terms of background job libraries, there's also a lot of apps still using DelayedJob and others that "can't" upgrade to something better such as Sidekiq due to the sheer amount of work required. Standardizing on an API for this will help a lot and enable easily moving to the Latest and Greatest™ queueing library.

kalleth 12y ago

Sigh. This is like that timecop thing -- dumping more stuff in core that lives better as part of a gem. Resque, DelayedJob, Sidekiq... All have their drawbacks/benefits. I know this can just work as a 'wrapper', but in an agency environment, i've used queueing on about 20% of sites. Absolutely no reason to want to put this in core imo.

unfunco 12y ago
It being included will have no performance overhead since it's only loaded if it's used. I'm personally for this idea, I've used DJ, Resque and Sidekiq, all in the same project (increasing load), and in that order; It would have been much nicer to have to only change the Gemfile when I made the switch. Your comment, whilst a perfectly valid concern, reminds me of an article DHH posted: http://david.heinemeierhansson.com/2012/rails-is-omakase.html http://david.heinemeierhansson.com/2012/rails-is-omakase.htm...
- kalleth 12y ago
  Performance wasn't my concern at all. Overcomplicating the rails 'core' is my concern. I'd much rather start from a lightweight 'base' which solves most of the common cases and pull in extras as required for what i'm working on. Rails should be a framework that allows you to layer extra functionality ontop of it.
  - _pius 12y ago
    Overcomplicating the rails 'core' is my concern. I'd much rather start from a lightweight 'base' which solves most of the common cases ... Background jobs are a common case. How many web applications never send an e-mail?
    - mackwic 12y ago
      He probably sends emails synchronously.
amorphid 12y ago

dpeck 12y ago

I don't really see the need for this. A unified interface could be nice, but this will also limit the scope that background processors will operate in. Besides that the amount of rails projects that need background jobs aren't that large of a percentage comparatively. I'd rather see some more interesting things going into core, such as https://github.com/kickstarter/rack-attack https://github.com/kickstarter/rack-attack, that should be used by nearly every app.

timtas 12y ago

Having recently migrated from Resque to Sidekiq, I can attest that this solves a problem, albeit not terribly a big one. As with any abstraction, implementation-specific features make leaks. So far, the README doesn't say how I will control features like retries and queue weights, but maybe that's coming soon.