4 ms·

> Cron jobs are easy to write, but difficult to setup Please explain to me, how is cron hard to set up? (slightly of topic: if you didn't learn cron don't wor

by bostand 10y ago

> Cron jobs are easy to write, but difficult to setup Please explain to me, how is cron hard to set up? (slightly of topic: if you didn't learn cron don't worry. It is just a matter of time before it is replaced with some systemd service that only works half the time)

jcrites 10y ago
I'm not the author and didn't write that line, but I can speculate about the kinds of concerns he's imagining, looking at the situation from a DevOps/automation-focused point of view: How do you set up cron in such a way that your cron job runs on a machine somewhere, and will continue to do so for a long period of time? Sure, you can log into a machine and edit the crontab manually, but what will happen if that machine fails? Ka-blam, it suffered a hardware failure and is gone. Do you repeat your manual edits on another machine? (If someone else did this three years ago and is no longer with the organization, does anyone know what edits to reproduce?) OK so we need to build a mechanism that can ensure that at least one machine is running, and has this crontab installed on it. If the currently active machine fails, it needs to replace the machine and reinstall the crontab and software that the crontab runs. You need to monitor to detect when this happens, to kick that off, and you need to test the stack to be adequately sure it's going to work when it really does happen. There's infrastructure you can use to do that, of course, but it's all complexity to be mastered. With the approach the author's describing, one only needs to define configuration: what code to run on what schedule. It's declarative, and the infrastructure handles actually executing it on a machine somewhere, completely encapsulating the concerns related to "getting some hardware running" and "installing the code".
- bostand 10y ago
  > How do you set up cron in such a way that your cron job runs on a machine somewhere, and will continue to do so for a long period of time? Use supervisord? Or (god forgive me) systemd services?
  - jcrites 10y ago
    Cron itself is a system service and is not something I'd hope you would need to run under supervisord. Some of the challenges that I mentioned, though, are not just arranging the configuration that you want to have on the machine (whether cron or supervisord configuration), but getting the configuration onto the machine and ensuring it stays up and working long term. When looking at the problem with a long time horizon and larger system context, one expects any machine to fail and need to be replaced, so handling that is important. With this perspective, it doesn't matter how reliable your process supervisor is: the code can be written perfectly and the application will still fail to do its job if the machine it's running on halts! I don't want to have to care if my machines suffer a hardware failure; I just want a replacement to be brought online seamlessly. Sometimes in cases like this, one needs a whole lot of infrastructure to solve what feels like it ought to be a simple problem. In larger system contexts it can be quite inconvenient to invoke the full complexity needed for reliable servers just to run a job periodically. Once you start to tie the satisfaction of any business requirement to the successful completion of a cron job, lots of problems crop up. Even after you set that configuration up, there are still tricky issues to consider like: what if the host running the cron job fails at just the wrong time, like at 11:59 when the job starts at 12:00? Is it possible the job might not run at all today even if the host is replaced promptly? Is that OK, and if not what do we do about it? If you have many different cron jobs with slightly different security contexts and permission, then do you launch a separate machine for each one? Do you set them up to share the same server? How will you know if, a few years from now, that server doesn't have enough power to run all the jobs fast enough? Will you notice if it's gradually slowing down? These are some of the issues I'd consider if I was tackling a business problem that required doing something on a regular schedule. Serverless-style infrastructure takes away a lot of these concerns. There's nothing to host or maintain yourself; no servers needed at all: just the definition of the code you want to run, how frequently you want to run it, and what resources (if any) it needs access to. A serverless task that runs once today will continue to run indefinitely, and it doesn't cost more to set up than the naive crontab approach.

by bostand 10y ago

jcrites 10y ago

I'm not the author and didn't write that line, but I can speculate about the kinds of concerns he's imagining, looking at the situation from a DevOps/automation-focused point of view: How do you set up cron in such a way that your cron job runs on a machine somewhere, and will continue to do so for a long period of time? Sure, you can log into a machine and edit the crontab manually, but what will happen if that machine fails? Ka-blam, it suffered a hardware failure and is gone. Do you repeat your manual edits on another machine? (If someone else did this three years ago and is no longer with the organization, does anyone know what edits to reproduce?) OK so we need to build a mechanism that can ensure that at least one machine is running, and has this crontab installed on it. If the currently active machine fails, it needs to replace the machine and reinstall the crontab and software that the crontab runs. You need to monitor to detect when this happens, to kick that off, and you need to test the stack to be adequately sure it's going to work when it really does happen. There's infrastructure you can use to do that, of course, but it's all complexity to be mastered. With the approach the author's describing, one only needs to define configuration: what code to run on what schedule. It's declarative, and the infrastructure handles actually executing it on a machine somewhere, completely encapsulating the concerns related to "getting some hardware running" and "installing the code".

bostand 10y ago
> How do you set up cron in such a way that your cron job runs on a machine somewhere, and will continue to do so for a long period of time? Use supervisord? Or (god forgive me) systemd services?
- jcrites 10y ago
  Cron itself is a system service and is not something I'd hope you would need to run under supervisord. Some of the challenges that I mentioned, though, are not just arranging the configuration that you want to have on the machine (whether cron or supervisord configuration), but getting the configuration onto the machine and ensuring it stays up and working long term. When looking at the problem with a long time horizon and larger system context, one expects any machine to fail and need to be replaced, so handling that is important. With this perspective, it doesn't matter how reliable your process supervisor is: the code can be written perfectly and the application will still fail to do its job if the machine it's running on halts! I don't want to have to care if my machines suffer a hardware failure; I just want a replacement to be brought online seamlessly. Sometimes in cases like this, one needs a whole lot of infrastructure to solve what feels like it ought to be a simple problem. In larger system contexts it can be quite inconvenient to invoke the full complexity needed for reliable servers just to run a job periodically. Once you start to tie the satisfaction of any business requirement to the successful completion of a cron job, lots of problems crop up. Even after you set that configuration up, there are still tricky issues to consider like: what if the host running the cron job fails at just the wrong time, like at 11:59 when the job starts at 12:00? Is it possible the job might not run at all today even if the host is replaced promptly? Is that OK, and if not what do we do about it? If you have many different cron jobs with slightly different security contexts and permission, then do you launch a separate machine for each one? Do you set them up to share the same server? How will you know if, a few years from now, that server doesn't have enough power to run all the jobs fast enough? Will you notice if it's gradually slowing down? These are some of the issues I'd consider if I was tackling a business problem that required doing something on a regular schedule. Serverless-style infrastructure takes away a lot of these concerns. There's nothing to host or maintain yourself; no servers needed at all: just the definition of the code you want to run, how frequently you want to run it, and what resources (if any) it needs access to. A serverless task that runs once today will continue to run indefinitely, and it doesn't cost more to set up than the naive crontab approach.