4 ms·
Jenkins is honestly Really Good as a scheduling system. The best, frankly. If you can combine it with a library- or tool-style DAG system (vs. server-style lik
by slotrans 3y ago
Jenkins is honestly Really Good as a scheduling system. The best, frankly.
If you can combine it with a library- or tool-style DAG system (vs. server-style like Airflow), like make or Luigi or NextFlow or even Step Functions, that is a great sweet spot.
- dharmab 3y agoIn my experience Jenkins is like shell scripting- in the hands of hands of someone who understands its strengths and weaknesses and is very disciplined in how it is used and maintained, it's both performant and flexible. If you follow the path of least resistance it becomes a mess.
- patrick451 3y agoI agree. For some reason, Airflow really sucks at running something on a cron-like timer. I can't remember the all issues I had with it exactly (this was several jobs ago), but getting it to run a job at the same time everyday was a nightmare. One issue I do recall was that it treated the first run differently, basically ignoring your defined schedule. And I think "first run" meant every time Airflow was restarted. So if Bad Things happened if that job ran the wrong time of day, you had to add extra logic into the job itself to abort if it was being called at the wrong time. How they managed to make this so difficult is mind boggling. By contrast, Jenkins will reliably execute a job on a timer without issues.