5 ms·

Jobber: A replacement for cron, with status-reporting and error-handling

35 points by dshearer 12y ago

faragon 12y ago
KCSS (Keep Cron Simple, Stupid)
- coldtea 12y ago
  Why? Because "change is bad, mkkkay"? It's not like Cron covers all our needs... The "do one simple thing and do it well" principle worked best for the simple needs of late '70s OSes. In 2014 it ends up with MORE (not LESS) complexity, built on top of those simple tools to coordinate them to do anything useful. And of course they're not even that smart in how they work together... Like "make" has grown configure and autoconf and all that cruft (because we "kept it simple").
  - faragon 12y ago
    Cron already covers all my job scheduling needs, yes. You can do all the "recover thing" in the scripts called by cron, without the need of complicating 99% of cases that don't need such complication. "Keep it simple, stupid" is still valid for 2015, in my opinion. About your off-topic comment: autoconf/automake adds crazy complexity. I avoid it as much as possible, using just Make (it allows you context detection and lots of conditional handling, without complicating things).
  - feld 12y ago
    autoconf is insane. Just write your own configure script... Keeping it simple does not need to cause abominations like autoconf. I was glad to see the new ntimed project is not using it. Here's a comment from the custom configure script: # Handwritten configure script. # ============================= # # Before you suggest I use tools for this, please read: # # https://www.varnishcache.org/docs/trunk/phk/autocrap.html # and # http://queue.acm.org/detail.cfm?id=2349257 # from https://github.com/bsdphk/Ntimed/blob/master/configure https://github.com/bsdphk/Ntimed/blob/master/configure
  - sre_ops 12y ago
    Yes, change for the sake of change is bad. Cron has a couple of problem - - 1 is fine-grained scheduling. - 2 ability to set jobs to "catchup" or to skip on failure. Cron has no other real problems. Complexity is always bad. Always.
ryansouza 12y ago
"When discussing jobs, by “job error” we mean the failure of a particular run of a job, whereas by “job failure” we mean a job that jobber has decided not to schedule anymore due to one or more recent job errors."
laurencei 12y ago
Congrats on shipping. Looks interesting - great idea to replace 'cron' itself with a better implementation. (Shameless plug) - for those people that have to stick with cron (and cant switch to Jobber) and need status report and error handling I have a SaaS that handles this. See my bio for details.
moe 12y ago
https://github.com/dshearer/jobber/blob/master/jobber/jobber.go#L52 https://github.com/dshearer/jobber/blob/master/jobber/jobber... No thanks.
- rasur 12y ago
  Yeah, I'm with you on that TBH. There's nothing like a good bit of condescension from a developer to really win over their users. It was sad a few decades ago, and it's still sad now.
- akerl_ 12y ago
  Meh. If this was indicative of anything other than somebody wanting to have a bit of fun when typing error messages, you might be on to something. I don't mind a bit of humanity in my soul-less machines
  - jacquesm 12y ago
    Think of it as a message from the writer to the user.
    - akerl_ 12y ago
      I am thinking of it as a message from the writer to the user, and doing so gives me a little chuckle as I remember there is a human on the other side of the code that makes my server run.
      - jacquesm 12y ago
        It's funny, the first time, maybe. But if you use software all the time then such 'funnyness' becomes irritating. I'm trying to imagine working hard on trying to solve some issue for someone, mistyping a command and getting crap like that thrown back at me. Imagine all your software would return its errors laced with profanity. If you write code for yourself then that's one thing, as soon as you write code for potential distribution to a large audience (cron is present on just about any unix machine) then I think you should restrain yourself from trying to be funny (or at a minimum from having your software by proxy call people names). Call me old fashioned, it just doesn't work for me.
ukandy 12y ago
Recovery of third party software seems to be the USP here (logging and reporting aren't hard to achieve normally). I think I would have made a script to sit in-between, standard cron and third party software to solve the recovery problem where needed.
SwellJoe 12y ago
I suspect we are going to see some upheaval in core services in the coming years...I'll be curious to see if systemd.timer takes the place of cron or if something else does. There is a good argument to be made for at least updating cron. Link for systemd.timer: http://www.freedesktop.org/software/systemd/man/systemd.timer.html http://www.freedesktop.org/software/systemd/man/systemd.time... And some discussion of using it to replace cron: https://wiki.archlinux.org/index.php/Systemd/Timers#As_a_cron_replacement https://wiki.archlinux.org/index.php/Systemd/Timers#As_a_cro...
vinceguidry 12y ago
If you're using Ruby, whenever is a better solution. https://github.com/javan/whenever https://github.com/javan/whenever Wraps cron so you don't have to. Integrates with Capistrano.
_nato_ 12y ago
Nice to see some activity in this space. gives me a little encouragement for my lil up-start. Maybe we could trade notes? http://ikura.co http://ikura.co
- dshearer 12y ago
  Best of luck on your startup! It looks like your goals for Ikura are more ambitious than mine for Jobber --- e.g., Jobber doesn't spawn multiple instances of a job like Ikura does.
mikikian 12y ago
This looks very cool. Congrats on shipping. As a non-developer, (honest question) how does this compare to Airbnb's Chronos? I noticed you used Golang, and I'm guessing they have a better frontend. They taut "managed distributed solution". Any other differences?
- dshearer 12y ago
  Thanks for the congrats! Actually, it's not shipping quite yet --- it's still in beta. Jobber is really just cron + support for error-recovery strategies like exponential backoff + support for getting the job execution history. It is very simple to deploy: it consists of just two (native) binaries, and has no dependencies (well, other than Bash). Chronos also supports these additional features (although I don't know about exponential backoff), but Chronos is much more ambitious than Jobber. Probably the biggest differences are that Chronos supports farming jobs out to other nodes and that it supports dependencies between jobs. And of course Chronos has a nice GUI. On the other hand, I imagine it's a bigger deal to deploy, although I haven't done so myself. So, if you have a bunch of related jobs that need to run on multiple nodes, Chronos would probably be best. Otherwise, try Jobber!

35 points by dshearer 12y ago

faragon 12y ago

KCSS (Keep Cron Simple, Stupid)

coldtea 12y ago
Why? Because "change is bad, mkkkay"? It's not like Cron covers all our needs... The "do one simple thing and do it well" principle worked best for the simple needs of late '70s OSes. In 2014 it ends up with MORE (not LESS) complexity, built on top of those simple tools to coordinate them to do anything useful. And of course they're not even that smart in how they work together... Like "make" has grown configure and autoconf and all that cruft (because we "kept it simple").
- faragon 12y ago
  Cron already covers all my job scheduling needs, yes. You can do all the "recover thing" in the scripts called by cron, without the need of complicating 99% of cases that don't need such complication. "Keep it simple, stupid" is still valid for 2015, in my opinion. About your off-topic comment: autoconf/automake adds crazy complexity. I avoid it as much as possible, using just Make (it allows you context detection and lots of conditional handling, without complicating things).
- feld 12y ago
  autoconf is insane. Just write your own configure script... Keeping it simple does not need to cause abominations like autoconf. I was glad to see the new ntimed project is not using it. Here's a comment from the custom configure script: # Handwritten configure script. # ============================= # # Before you suggest I use tools for this, please read: # # https://www.varnishcache.org/docs/trunk/phk/autocrap.html # and # http://queue.acm.org/detail.cfm?id=2349257 # from https://github.com/bsdphk/Ntimed/blob/master/configure https://github.com/bsdphk/Ntimed/blob/master/configure
- sre_ops 12y ago
  Yes, change for the sake of change is bad. Cron has a couple of problem - - 1 is fine-grained scheduling. - 2 ability to set jobs to "catchup" or to skip on failure. Cron has no other real problems. Complexity is always bad. Always.

ryansouza 12y ago

"When discussing jobs, by “job error” we mean the failure of a particular run of a job, whereas by “job failure” we mean a job that jobber has decided not to schedule anymore due to one or more recent job errors."

laurencei 12y ago

Congrats on shipping. Looks interesting - great idea to replace 'cron' itself with a better implementation. (Shameless plug) - for those people that have to stick with cron (and cant switch to Jobber) and need status report and error handling I have a SaaS that handles this. See my bio for details.

moe 12y ago

https://github.com/dshearer/jobber/blob/master/jobber/jobber.go#L52 https://github.com/dshearer/jobber/blob/master/jobber/jobber... No thanks.

rasur 12y ago
Yeah, I'm with you on that TBH. There's nothing like a good bit of condescension from a developer to really win over their users. It was sad a few decades ago, and it's still sad now.
akerl_ 12y ago
Meh. If this was indicative of anything other than somebody wanting to have a bit of fun when typing error messages, you might be on to something. I don't mind a bit of humanity in my soul-less machines
- jacquesm 12y ago
  Think of it as a message from the writer to the user.
  - akerl_ 12y ago
    I am thinking of it as a message from the writer to the user, and doing so gives me a little chuckle as I remember there is a human on the other side of the code that makes my server run.
    - jacquesm 12y ago
      It's funny, the first time, maybe. But if you use software all the time then such 'funnyness' becomes irritating. I'm trying to imagine working hard on trying to solve some issue for someone, mistyping a command and getting crap like that thrown back at me. Imagine all your software would return its errors laced with profanity. If you write code for yourself then that's one thing, as soon as you write code for potential distribution to a large audience (cron is present on just about any unix machine) then I think you should restrain yourself from trying to be funny (or at a minimum from having your software by proxy call people names). Call me old fashioned, it just doesn't work for me.

ukandy 12y ago

Recovery of third party software seems to be the USP here (logging and reporting aren't hard to achieve normally). I think I would have made a script to sit in-between, standard cron and third party software to solve the recovery problem where needed.

SwellJoe 12y ago

I suspect we are going to see some upheaval in core services in the coming years...I'll be curious to see if systemd.timer takes the place of cron or if something else does. There is a good argument to be made for at least updating cron. Link for systemd.timer: http://www.freedesktop.org/software/systemd/man/systemd.timer.html http://www.freedesktop.org/software/systemd/man/systemd.time... And some discussion of using it to replace cron: https://wiki.archlinux.org/index.php/Systemd/Timers#As_a_cron_replacement https://wiki.archlinux.org/index.php/Systemd/Timers#As_a_cro...

vinceguidry 12y ago

If you're using Ruby, whenever is a better solution. https://github.com/javan/whenever https://github.com/javan/whenever Wraps cron so you don't have to. Integrates with Capistrano.

_nato_ 12y ago

Nice to see some activity in this space. gives me a little encouragement for my lil up-start. Maybe we could trade notes? http://ikura.co http://ikura.co

dshearer 12y ago
Best of luck on your startup! It looks like your goals for Ikura are more ambitious than mine for Jobber --- e.g., Jobber doesn't spawn multiple instances of a job like Ikura does.

mikikian 12y ago

This looks very cool. Congrats on shipping. As a non-developer, (honest question) how does this compare to Airbnb's Chronos? I noticed you used Golang, and I'm guessing they have a better frontend. They taut "managed distributed solution". Any other differences?

dshearer 12y ago
Thanks for the congrats! Actually, it's not shipping quite yet --- it's still in beta. Jobber is really just cron + support for error-recovery strategies like exponential backoff + support for getting the job execution history. It is very simple to deploy: it consists of just two (native) binaries, and has no dependencies (well, other than Bash). Chronos also supports these additional features (although I don't know about exponential backoff), but Chronos is much more ambitious than Jobber. Probably the biggest differences are that Chronos supports farming jobs out to other nodes and that it supports dependencies between jobs. And of course Chronos has a nice GUI. On the other hand, I imagine it's a bigger deal to deploy, although I haven't done so myself. So, if you have a bunch of related jobs that need to run on multiple nodes, Chronos would probably be best. Otherwise, try Jobber!