5 ms·
How does it handle state and restarts? What happens if a job is scheduled to run "before 10am", then the entire server restarts at 9:55am, will it try to run th
by wmichelin 4y ago
How does it handle state and restarts? What happens if a job is scheduled to run "before 10am", then the entire server restarts at 9:55am, will it try to run that same job again when it boots back up?
- angrais 4y agoIf it took less than 5 minutes to boot then I can't see why it wouldn't work. How would that work for other schedulers? Also, if a server reboots that's quite bad all round anyway. Hopefully you'd be notified directly.
- mplewis 4y agoOther schedulers have a durable database of attempted runs. This doesn't seem to have anything like that.
- dr_kiszonka 4y agoWould you have a recommendation for an easy to use Python scheduler with such a feature for a personal project?
- MonkeyMalarky 4y agoSeconding this. I would love a middle tier job scheduler / manager for Python that has persistence. I feel like there's a missing middle between cron+scripts and the enterprise grade tooling built for ETL tasks.
- staticautomatic 4y agoAPScheduler
- dr_kiszonka 4y agoIt looks pretty good. Thanks!
- lobocinza 4y agoI think you can create ephemeral timers with Systemd if you're on Linux.
- tekromancr 4y agoCelery Beat is pretty good
- beci 4y agohttps://www.prefect.io/ https://www.prefect.io/ is also pretty easy.
- lovelearning 4y agoFrom its docs, I understand that it has a task status logging layer[1] which can persist to SQL/Mongo (using another framework called Red Bird[2]). I haven't tried them out though. [1]: https://red-engine.readthedocs.io/en/stable/tutorial/basic.html#changing-logging-destination https://red-engine.readthedocs.io/en/stable/tutorial/basic.h... [2]: https://red-bird.readthedocs.io/en/latest/ https://red-bird.readthedocs.io/en/latest/
- vore 4y agoI think on the contrary, it's important to build software that's resilient to random reboots: you can't control things like losing power or your CPU failing, so the less toil you have to manage around that the better.
- Closi 4y agoIt will run it if the scheduler is called before 10am according to the docs - at runtime the conditions must be met.
- wmichelin 4y agoRight but what if it already ran? Should the jobs be written such that they are tolerant to re-runs?
- falcor84 4y agoI think that regardless of the scheduling tool, it is very worthwhile putting in the effort to make your tasks idempotent. I've had tons of cases when I had to repeatedly rerun failed tasks "in anger", and was always grateful to know I made it safe to do so.