3 ms·
Look, don't get me wrong, cron is a super useful tool for doing system-level tasks on a schedule but "grep a timestamp and run a program if it matches" is pract
by Spivak 7y ago
Look, don't get me wrong, cron is a super useful tool for doing system-level tasks on a schedule but "grep a timestamp and run a program if it matches" is practically the definition of "doesn't scale."
Unless you code it yourself in your program cron doesn't...
- understand dependencies
- run jobs across multiple hosts
- handle errors (e.g. if err run this)
- do any branching whatsoever
- handle timespecs with more granularity than 1m
- do any reporting (mail counts-ish but if you're telling me you have an email based automation flow I won't believe you)
- do any metrics collection
- run pools of jobs
- understand priorities
- allow you to queue jobs
The value prop of things like airflow is that they factor all this logic out of your applications. At the point where you need these things "run a thing at a specific time" is the most trivial part.
- cat199 7y ago> - understand priorities > - allow you to queue jobs technically this can be done oldschool with cron+at/batch.
- blowski 7y agoI think you should write their homepage as your comment is much more informative.
- ahbyb 7y agoThen it's the wrong comparison, because cron simply schedules other commands, it's not a programming language.
- dragonwriter 7y agoYes, chron is just a very simple, single-node, low-time-resolution, system-level scheduler. While it may be good at that, that is very often (always would be an exaggeration, probably) not a great tool to be used (at least as an exposed component) in automation of modern production systems.
- dredmorbius 7y agoCron is a scheduler. If you need other features, your cron script(s) themselves can check those. Addressing your specific items: Dependencies. Depending on which of these you're referencing, the package manager (which installs a given cron script), configuration manager, or script itself, can and should test and check these. Multiple hosts. There are numerous multi-host utilities, ranging from SSH extensions / utilities to full multi-host configuration tools (Ansible, Salt, Chef, cfengine, Puppet, etc.) At heart, this isn't cron's problem. It's a schedular, not an orchestrator. Error handling. Your script's issue, not crons. Cron is a schedular, not an error handler. Branching. Not sure what you're referencing, but again, the task of the script, generally. Sub-minute granularity. Cron is a schedular, not a daemon substitute. If you need a daemon operating at sub-minute granularity, write a daemon. Reporting. This is the job of your logging facility, generally. Metrics. This is the job of your accounting facility: systat, syslog, etc., etc. Pools of jobs. This is the job of your script(s), generally. If you have a set or sequence of tasks, trigger the master with cron, and have it manage the set. Again, cron is a scheduler, not an orchestrator. Priorities. Nice. Queue jobs. You want 'at' or 'batch', not cron. Cron is a scheuler, not a job queue.