7 ms·

In search of a better job scheduler

112 points by K2L8M11N2 7y ago

mwielbut 7y ago
One more: Rundeck https://docs.rundeck.com/docs/manual/ https://docs.rundeck.com/docs/manual/
- mjw1007 7y ago
  Also https://github.com/brutusin/wava https://github.com/brutusin/wava
gravypod 7y ago
I'm pretty sure something like Kube could be used to do this using CronJobs, pod affinities, and Prometheus+AlertManager for alerts and notifications. You can specify memory, CPU, retries, monitor, and run on many computers. I think Kube is like systemd + cron + containers had a baby.
halayli 7y ago
I know this is mac specific, but since op is looking for personal infra level tooling/scheduling, there might be an equivalent what ical offers: create ical event+repeat+alert=Custom(Open file) this will open a file every repeat event. The file can be shell/workflow or whatever you want to run.
mjw1007 7y ago
« better specs for jobs dependencies, timeouts, resource policies and retries without hacky wrappers and boilerplate » This is what I miss. There's a strange gap in traditional Unix there, compared to other time-sharing systems. I have a feeling that if Unix had had a primitive job manager then by 1990 there would have been a pile of Gnu extensions and by 2000 there would have been people reinventing it, and GUIs that looked a bit like what you get to monitor print queues.
kinow 7y ago
I work on a meta scheduler that solves the problem of periodicity on cron - https://cylc.github.io/ https://cylc.github.io/ It was created to manage weather forecast model runs. The new version which we are working on will have a simpler web GUI to replace its old PyGTK and Py2. It handles o ly scheduling, and supports cyclic graphs (contrary to most workflow managers build with DAGs only). No Python API yet, but coming in future release. So only a custom suite.rc INI style file that supports Jinja2 too.
rb808 7y ago
Anyone have a good open source replacement for Autosys or controlM? Should be easy to replace but every enterprise I've seen still pays for one of those.
- dstroot 7y ago
  This! I am highly interested if anyone has any suggestions.
longwave 7y ago
Jenkins works reasonably well as a job scheduler. The UI is functional if not pretty, it is fairly easy to configure and use even for a large number of jobs, and the plugin ecosystem allows you to extend it in many ways without writing any code.
- baq 7y ago
  Any decent CI server should do, though exponential back off isn’t something that I’m aware can be configured without custom scripting.
- pm90 7y ago
  If the only way to get a decent cron is to run a full blown Jenkins server it is time I quit the tech industry.
- hinkley 7y ago
  It would be kinda nice if we could pull the job runner out as it’s own thing.
thesorrow 7y ago
You can create systemd timers and services quite easily with NixOS configuration : https://nixos.wiki/wiki/Nix_Cookbook https://nixos.wiki/wiki/Nix_Cookbook
- Mathnerd314 7y ago
  The issue AFAICT is that you need to define a separate unit to run the command. It's not clear why systemd has this restriction on timers, other than implementation convenience.
9dev 7y ago
I don't get the systemd hate in this specific case. Their timers are pretty much the best possible option you have to schedule jobs, hands down. No cron Daemon is ever going to get there due to the lack of deep integration with the rest of the init system. I know it takes a while to get used to the ini file madness, but honestly, I can't detect so much boilerplate there: every single line in my units and timers makes sense. If you're not familiar with that system yet, I can only encourage you to at least take a look at systemd timers. They are really cool! :)
- karlicoss 7y ago
  Hey, author here! Hope it doesn't read like I meant hate to systemd, please let me know where if you feel like I did! I guess some of the cons I listed count as subjective, but I doubt anyone would argue that adjusting a systemd job is easier than cron job :) Also I emphasised that I had non-tecnhical user in mind, who would definitely have hard time understanding what are 'timers', and why their job type has to be 'Simple'. But I agree that forcing cron into doing what it wasn't meant for isn't probably going to go anywhere. That's why I suggested that using systemd as a backend is probably the most reasonable option after all, if only there was a friendlier tool to make the process as simple as with cron.
  - arminiusreturns 7y ago
    I think you should check out mcron, which is basically cron redone but in GNU guile (scheme) with more features. I think it fails some of your requirements (the syntax can be a bit more to wrap ones head around than standard cron), but it is powerful and seems promising. https://www.gnu.org/software/mcron/manual/mcron.html#Introduction https://www.gnu.org/software/mcron/manual/mcron.html#Introdu...
- CameronNemo 7y ago
  I prefer snooze over cron for similar reasons. It is a transparent chain loaded command that you can integrate win any service supervisor. https://github.com/leahneukirchen/snooze https://github.com/leahneukirchen/snooze
- 7y ago
dswalter 7y ago
Is there a reason not to consider apache airflow? https://airflow.apache.org/ https://airflow.apache.org/ You could run it on a digital ocean droplet so you don't have to worry about your laptop turning off. It covers retry and cron-style job kickoffs.
- demosito666 7y ago
  Complexity, effectively not supporting shorter scheduling interval (if you need to run something more frequently than every 20 minutes, beware), some bugs don't get fixed for years and are very hard to debug. We ended up restating the scheduler on regular Cron to kick stale task queue.
defanor 7y ago
I keep wondering whether there's a job scheduler for me too, but not finding much. At work I'm using/writing a custom one: with multiple schedules in crontab format for each task, frequent automated task updates (by an identifier) from multiple sources, additional flags for some jobs to not run if the parameters didn't change, and/or to rerun every few hours to ensure that proper states are set. It still felt like a generic enough task to use an external tool with those requirements, but then additional functionality appeared, such as changing task parameters according to certain rules (and internal state). At this point it seems that the way to go is either a custom scheduler or just regular cron, crontab editing, and wrappers akin to cronic (or otherwise handling the state-related configuration in the tasks it runs) -- that is, also a custom scheduling system, but incorporating cron. Here cron's simplicity seems fine: say, prevention of duplicate tasks should be by those "task group" IDs, rather than just jobs listed in crontab (so even if cron had the latter, it wouldn't be useful).
encoderer 7y ago
I love all the options we have today. At Cronitor we are seeing a lot of users migrating to Airflow, Kubernetes cron jobs, and scheduled lambdas. Like databases, different workloads will benefit from different guarantees. Look for the right tool for the job, not the right tool for all time.
- demosito666 7y ago
  As someone migrating from airflow back to cron jobs (I can do that doesn't really need to be distributed), I can't agree more. And the important thing is to look at the complexity of the solution as well.
darkteflon 7y ago
Slightly tangential but all I could think about while reading this was: Ansible. I switched to it about 6 months back - including for managing e.g. cron jobs and systemd - and I can’t imagine going back to doing any of this by hand. High-level abstraction and jinja2 templating makes this stuff a breeze.
1337shadow 7y ago
I automated the tedious work to create systemd timers in an ansible role: https://yourlabs.io/oss/yourlabs.timer https://yourlabs.io/oss/yourlabs.timer As a result, I can execute it include it in another ansible role, or use bigsudo to apply it as a one-off command: bigsudo yourlabs.timer @somehost name=your-backup cmd=/your/backup.sh oncalendar='*-*-* 00:00:00 Europe/Paris'
talelcool 7y ago
Drone.io for me, simple and dockerized

112 points by K2L8M11N2 7y ago

mwielbut 7y ago

One more: Rundeck https://docs.rundeck.com/docs/manual/ https://docs.rundeck.com/docs/manual/

mjw1007 7y ago
Also https://github.com/brutusin/wava https://github.com/brutusin/wava

gravypod 7y ago

I'm pretty sure something like Kube could be used to do this using CronJobs, pod affinities, and Prometheus+AlertManager for alerts and notifications. You can specify memory, CPU, retries, monitor, and run on many computers. I think Kube is like systemd + cron + containers had a baby.

halayli 7y ago

I know this is mac specific, but since op is looking for personal infra level tooling/scheduling, there might be an equivalent what ical offers: create ical event+repeat+alert=Custom(Open file) this will open a file every repeat event. The file can be shell/workflow or whatever you want to run.

mjw1007 7y ago

« better specs for jobs dependencies, timeouts, resource policies and retries without hacky wrappers and boilerplate » This is what I miss. There's a strange gap in traditional Unix there, compared to other time-sharing systems. I have a feeling that if Unix had had a primitive job manager then by 1990 there would have been a pile of Gnu extensions and by 2000 there would have been people reinventing it, and GUIs that looked a bit like what you get to monitor print queues.

kinow 7y ago

I work on a meta scheduler that solves the problem of periodicity on cron - https://cylc.github.io/ https://cylc.github.io/ It was created to manage weather forecast model runs. The new version which we are working on will have a simpler web GUI to replace its old PyGTK and Py2. It handles o ly scheduling, and supports cyclic graphs (contrary to most workflow managers build with DAGs only). No Python API yet, but coming in future release. So only a custom suite.rc INI style file that supports Jinja2 too.

rb808 7y ago

Anyone have a good open source replacement for Autosys or controlM? Should be easy to replace but every enterprise I've seen still pays for one of those.

dstroot 7y ago
This! I am highly interested if anyone has any suggestions.

longwave 7y ago

Jenkins works reasonably well as a job scheduler. The UI is functional if not pretty, it is fairly easy to configure and use even for a large number of jobs, and the plugin ecosystem allows you to extend it in many ways without writing any code.

baq 7y ago
Any decent CI server should do, though exponential back off isn’t something that I’m aware can be configured without custom scripting.
pm90 7y ago
If the only way to get a decent cron is to run a full blown Jenkins server it is time I quit the tech industry.
hinkley 7y ago
It would be kinda nice if we could pull the job runner out as it’s own thing.

thesorrow 7y ago

You can create systemd timers and services quite easily with NixOS configuration : https://nixos.wiki/wiki/Nix_Cookbook https://nixos.wiki/wiki/Nix_Cookbook

Mathnerd314 7y ago
The issue AFAICT is that you need to define a separate unit to run the command. It's not clear why systemd has this restriction on timers, other than implementation convenience.

9dev 7y ago

I don't get the systemd hate in this specific case. Their timers are pretty much the best possible option you have to schedule jobs, hands down. No cron Daemon is ever going to get there due to the lack of deep integration with the rest of the init system. I know it takes a while to get used to the ini file madness, but honestly, I can't detect so much boilerplate there: every single line in my units and timers makes sense. If you're not familiar with that system yet, I can only encourage you to at least take a look at systemd timers. They are really cool! :)

karlicoss 7y ago
Hey, author here! Hope it doesn't read like I meant hate to systemd, please let me know where if you feel like I did! I guess some of the cons I listed count as subjective, but I doubt anyone would argue that adjusting a systemd job is easier than cron job :) Also I emphasised that I had non-tecnhical user in mind, who would definitely have hard time understanding what are 'timers', and why their job type has to be 'Simple'. But I agree that forcing cron into doing what it wasn't meant for isn't probably going to go anywhere. That's why I suggested that using systemd as a backend is probably the most reasonable option after all, if only there was a friendlier tool to make the process as simple as with cron.
- arminiusreturns 7y ago
  I think you should check out mcron, which is basically cron redone but in GNU guile (scheme) with more features. I think it fails some of your requirements (the syntax can be a bit more to wrap ones head around than standard cron), but it is powerful and seems promising. https://www.gnu.org/software/mcron/manual/mcron.html#Introduction https://www.gnu.org/software/mcron/manual/mcron.html#Introdu...
CameronNemo 7y ago
I prefer snooze over cron for similar reasons. It is a transparent chain loaded command that you can integrate win any service supervisor. https://github.com/leahneukirchen/snooze https://github.com/leahneukirchen/snooze
7y ago

dswalter 7y ago

Is there a reason not to consider apache airflow? https://airflow.apache.org/ https://airflow.apache.org/ You could run it on a digital ocean droplet so you don't have to worry about your laptop turning off. It covers retry and cron-style job kickoffs.

demosito666 7y ago
Complexity, effectively not supporting shorter scheduling interval (if you need to run something more frequently than every 20 minutes, beware), some bugs don't get fixed for years and are very hard to debug. We ended up restating the scheduler on regular Cron to kick stale task queue.

defanor 7y ago

I keep wondering whether there's a job scheduler for me too, but not finding much. At work I'm using/writing a custom one: with multiple schedules in crontab format for each task, frequent automated task updates (by an identifier) from multiple sources, additional flags for some jobs to not run if the parameters didn't change, and/or to rerun every few hours to ensure that proper states are set. It still felt like a generic enough task to use an external tool with those requirements, but then additional functionality appeared, such as changing task parameters according to certain rules (and internal state). At this point it seems that the way to go is either a custom scheduler or just regular cron, crontab editing, and wrappers akin to cronic (or otherwise handling the state-related configuration in the tasks it runs) -- that is, also a custom scheduling system, but incorporating cron. Here cron's simplicity seems fine: say, prevention of duplicate tasks should be by those "task group" IDs, rather than just jobs listed in crontab (so even if cron had the latter, it wouldn't be useful).

encoderer 7y ago

I love all the options we have today. At Cronitor we are seeing a lot of users migrating to Airflow, Kubernetes cron jobs, and scheduled lambdas. Like databases, different workloads will benefit from different guarantees. Look for the right tool for the job, not the right tool for all time.

demosito666 7y ago
As someone migrating from airflow back to cron jobs (I can do that doesn't really need to be distributed), I can't agree more. And the important thing is to look at the complexity of the solution as well.

darkteflon 7y ago

Slightly tangential but all I could think about while reading this was: Ansible. I switched to it about 6 months back - including for managing e.g. cron jobs and systemd - and I can’t imagine going back to doing any of this by hand. High-level abstraction and jinja2 templating makes this stuff a breeze.

1337shadow 7y ago

I automated the tedious work to create systemd timers in an ansible role: https://yourlabs.io/oss/yourlabs.timer https://yourlabs.io/oss/yourlabs.timer As a result, I can execute it include it in another ansible role, or use bigsudo to apply it as a one-off command: bigsudo yourlabs.timer @somehost name=your-backup cmd=/your/backup.sh oncalendar='*-*-* 00:00:00 Europe/Paris'

talelcool 7y ago

Drone.io for me, simple and dockerized