5 ms·

so, i'm not a very good engineer so maybe this is a stupid question... but i am going to ask it anyways. Cron is hard. Not in the sense that its challenging t

so, i'm not a very good engineer so maybe this is a stupid question... but i am going to ask it anyways. Cron is hard. Not in the sense that its challenging to set up but its a challenge to maintain and have visibility into. It seems like you need a whole set of infrastructure just to feel confident that things are running as intended. This has always lead me down the path of building a task manager type of micro-service running on an independent server to manage everything. But by the time im done, it feels like over kill. I just don't know what to do differently. Am I missing something?

wmfiv 10y ago
Something like http://www.easybatch.org/ http://www.easybatch.org/ may provide quite a bit of the infrastructure you're looking for. I'm sure there are equivalents for other languages.
chrisp_dc 10y ago
Sounds like you might be better off using Jenkins than roll your own. I use Jenkins over cron when I want easier auditing & authentication. I got the idea to use Jenkins as a 1-size fits all hammer from https://www.cloudbees.com/blog/drop-cron-use-jenkins-instead-scheduled-jobs https://www.cloudbees.com/blog/drop-cron-use-jenkins-instead...
- philip1209 10y ago
  I've done this many times in the past. Add an authenticated "cron" endpoint, and have a jenkins job curl it every X seconds. It's not perfect, but it's an easy way to see log, alert Pagerduty, etc.
- mickael-kerjean 10y ago
  Great to see I'm not the only one using Jenkins to look after crown jobs! However, it's to me way more than simply auditing & auth. The main advantage I found with Jenkins are: - you get a central place for all the cron jobs. That comes with all the advantages the most important one being nobody has to remember on which server is this freaking script - making an update can easily be done - running the task can even be done by non technical people. - easy backup - when a script become deprecated you can easily remove it and you're not very likely to let it there running for nothing forever - easy documentation for you tasks There's probably more but Jenkins is definitely awesome for cron jobs
- flukus 10y ago
  Using a CI server as a scheduler? How is that at all the right tool for the job?
philip1209 10y ago
I've been burned by bad cron monitoring many times in the past. Short answer: check out https://cronitor.io/ https://cronitor.io/. Basically, finish every cron job by telling their server that the job finished. It's good enough monitoring for most cases.

wmfiv 10y ago

Something like http://www.easybatch.org/ http://www.easybatch.org/ may provide quite a bit of the infrastructure you're looking for. I'm sure there are equivalents for other languages.

chrisp_dc 10y ago

Sounds like you might be better off using Jenkins than roll your own. I use Jenkins over cron when I want easier auditing & authentication. I got the idea to use Jenkins as a 1-size fits all hammer from https://www.cloudbees.com/blog/drop-cron-use-jenkins-instead-scheduled-jobs https://www.cloudbees.com/blog/drop-cron-use-jenkins-instead...

philip1209 10y ago
I've done this many times in the past. Add an authenticated "cron" endpoint, and have a jenkins job curl it every X seconds. It's not perfect, but it's an easy way to see log, alert Pagerduty, etc.
mickael-kerjean 10y ago
Great to see I'm not the only one using Jenkins to look after crown jobs! However, it's to me way more than simply auditing & auth. The main advantage I found with Jenkins are: - you get a central place for all the cron jobs. That comes with all the advantages the most important one being nobody has to remember on which server is this freaking script - making an update can easily be done - running the task can even be done by non technical people. - easy backup - when a script become deprecated you can easily remove it and you're not very likely to let it there running for nothing forever - easy documentation for you tasks There's probably more but Jenkins is definitely awesome for cron jobs
flukus 10y ago
Using a CI server as a scheduler? How is that at all the right tool for the job?

philip1209 10y ago

I've been burned by bad cron monitoring many times in the past. Short answer: check out https://cronitor.io/ https://cronitor.io/. Basically, finish every cron job by telling their server that the job finished. It's good enough monitoring for most cases.