5 ms·

How to Run Background Jobs on Heroku for Free

34 points by duck 16y ago

usaar333 16y ago
Clever use of app engine to poll Heroku. :) <shameless plug> You might also want to check out PiCloud (http://www.picloud.com/product/#features http://www.picloud.com/product/#features), which offers low-cost pay by millisecond python-based crons</shameless plug>
po 16y ago
Reminds me a bit of the django "Poor Man's Cron" project: http://code.google.com/p/django-poormanscron/ http://code.google.com/p/django-poormanscron/ I couldn't think of a reason to go with it with projects like Celery around, but I guess if my host was trying to charge a lot for background processing, this would do it.
po 16y ago
Background processing and regular processing are ultimately the same thing. It doesn't make sense to me why Heroku would try to price discriminate around that feature.
- wuputah 16y ago
  They don't. They both cost 5¢/hr per process. The only difference is, you get 1 'foreground' worker for free (per application), but not any 'background' workers.
  - sabat 16y ago
    The only difference is, you get 1 'foreground' worker for free That's the differentiation.
    - sabat 16y ago
      BTW, don't get me wrong: Heroku is a standout service, and I use it as an example of how dramatically easier it is to start a technology business compared to even 10 years ago. I just have this little nitpicky complaint, that they ought to include one background process for free. The lack of it is actually blocking me from using Heroku for a startup idea at the moment. (I could always pay, sure, but when you're at the proof-of-concept stage, you're reluctant to put out even that much money.)
wmwong 16y ago
This is a nice little hack that I'll stick into my toolbox. The comment on the post about turning the workers on and off when needed is interesting as well. The commenter uses less than an hour on average per month using the method. I wonder if it's time consuming to do this though?
- aarongough 16y ago
  Time consuming from a development stand-point, or billable-time standpoint? Switching to the auto-scaling version of delayed_job on heroku took me about 20 minutes and it has worked flawlessly ever since. The repo is here: http://github.com/pedro/delayed_job/tree/autoscaling http://github.com/pedro/delayed_job/tree/autoscaling
wuputah 16y ago
Background workers only cost you money (5¢/hour) when they're running. It's prorated to the second. You can avoid running one constantly by using pedro's "autoscaling" branch, meaning you can run a few hours of background processing--which should equate to hundreds or thousands of jobs--every month for a few nickels. http://github.com/pedro/delayed_job/tree/autoscaling http://github.com/pedro/delayed_job/tree/autoscaling Pedro is a Heroku employee, so this use is totally fine by them, although the branch is not officially supported software. [I'm friends with a few guys there, but I don't work there.]
- aarongough 16y ago
  I was just coming to the thread to post the exact same thing. I'm using the auto-scaling version of delayed_job in production on two applications and it has worked absolutely flawlessly. As the jobs are only for sending emails in the background I don't think I've racked up more that $1 over the last month...
swombat 16y ago
Background job processing is so essential to any app that I've ever built, that the lack of suitable free background processing (whether via cron or DJ) effectively makes Heroku not have a free option for me. Even if I'm writing a little toy app, it will probably involve DJ and cron, so it will need to be hosted somewhere else than Heroku. (as for bigger apps, let's not get started... Heroku is pretty expensive at scale, not sure what they have that justifies it)
- dublinclontarf 16y ago
  Ease of use, Heroku is really REALLY easy to use compared to GAE. I was going to go for GAE for a project a few months ago, and decided on Heroku because I can just get straight to work instead of fapping about configuring things. On another, slightly related note you could use Mirah on GAE, a ruby like language. It's pretty neat.
dublinclontarf 16y ago
What about having another (free) Heroku app to do the background processing? Anyone tried that or would the Heroku guys kill such a thing?
mtraven 16y ago
Here's a very simple library for Heroku that makes delayed jobs switch on background workers only when necessary. http://github.com/mtravers/cheepnis http://github.com/mtravers/cheepnis

34 points by duck 16y ago

Clever use of app engine to poll Heroku. :) <shameless plug> You might also want to check out PiCloud (http://www.picloud.com/product/#features http://www.picloud.com/product/#features), which offers low-cost pay by millisecond python-based crons</shameless plug>

po 16y ago

Reminds me a bit of the django "Poor Man's Cron" project: http://code.google.com/p/django-poormanscron/ http://code.google.com/p/django-poormanscron/ I couldn't think of a reason to go with it with projects like Celery around, but I guess if my host was trying to charge a lot for background processing, this would do it.

po 16y ago

Background processing and regular processing are ultimately the same thing. It doesn't make sense to me why Heroku would try to price discriminate around that feature.

wuputah 16y ago
They don't. They both cost 5¢/hr per process. The only difference is, you get 1 'foreground' worker for free (per application), but not any 'background' workers.
- sabat 16y ago
  The only difference is, you get 1 'foreground' worker for free That's the differentiation.
  - sabat 16y ago
    BTW, don't get me wrong: Heroku is a standout service, and I use it as an example of how dramatically easier it is to start a technology business compared to even 10 years ago. I just have this little nitpicky complaint, that they ought to include one background process for free. The lack of it is actually blocking me from using Heroku for a startup idea at the moment. (I could always pay, sure, but when you're at the proof-of-concept stage, you're reluctant to put out even that much money.)

wmwong 16y ago

This is a nice little hack that I'll stick into my toolbox. The comment on the post about turning the workers on and off when needed is interesting as well. The commenter uses less than an hour on average per month using the method. I wonder if it's time consuming to do this though?

aarongough 16y ago
Time consuming from a development stand-point, or billable-time standpoint? Switching to the auto-scaling version of delayed_job on heroku took me about 20 minutes and it has worked flawlessly ever since. The repo is here: http://github.com/pedro/delayed_job/tree/autoscaling http://github.com/pedro/delayed_job/tree/autoscaling

wuputah 16y ago

Background workers only cost you money (5¢/hour) when they're running. It's prorated to the second. You can avoid running one constantly by using pedro's "autoscaling" branch, meaning you can run a few hours of background processing--which should equate to hundreds or thousands of jobs--every month for a few nickels. http://github.com/pedro/delayed_job/tree/autoscaling http://github.com/pedro/delayed_job/tree/autoscaling Pedro is a Heroku employee, so this use is totally fine by them, although the branch is not officially supported software. [I'm friends with a few guys there, but I don't work there.]

aarongough 16y ago
I was just coming to the thread to post the exact same thing. I'm using the auto-scaling version of delayed_job in production on two applications and it has worked absolutely flawlessly. As the jobs are only for sending emails in the background I don't think I've racked up more that $1 over the last month...

swombat 16y ago

Background job processing is so essential to any app that I've ever built, that the lack of suitable free background processing (whether via cron or DJ) effectively makes Heroku not have a free option for me. Even if I'm writing a little toy app, it will probably involve DJ and cron, so it will need to be hosted somewhere else than Heroku. (as for bigger apps, let's not get started... Heroku is pretty expensive at scale, not sure what they have that justifies it)

dublinclontarf 16y ago
Ease of use, Heroku is really REALLY easy to use compared to GAE. I was going to go for GAE for a project a few months ago, and decided on Heroku because I can just get straight to work instead of fapping about configuring things. On another, slightly related note you could use Mirah on GAE, a ruby like language. It's pretty neat.

dublinclontarf 16y ago

What about having another (free) Heroku app to do the background processing? Anyone tried that or would the Heroku guys kill such a thing?

mtraven 16y ago

Here's a very simple library for Heroku that makes delayed jobs switch on background workers only when necessary. http://github.com/mtravers/cheepnis http://github.com/mtravers/cheepnis