7 ms·

Cron works great when you don’t need to guarantee execution, e.g., if a server goes down. Unfortunately, all the alternatives are pretty heavyweight, e.g., Jenk

Cron works great when you don’t need to guarantee execution, e.g., if a server goes down. Unfortunately, all the alternatives are pretty heavyweight, e.g., Jenkins, Azkaban, Airflow. I’ve been working a job scheduler that strives to work like a distributed cron. It works with very little code, because it leans heavily on Postgres (for distributed locking, parsing time interval expressions, configuration storage, log storage) and PostgREST (for the HTTP API). The application binary (~100 lines of haskell), polls for new jobs, then checks out and execute tasks. The code is here if you’re interested: https://github.com/finix-payments/jobs https://github.com/finix-payments/jobs It compiles to machine code, so deploying the binary is easy. That said, I’d like to add some tooling to simplify deploying and configuring Postgres and PostgREST.

freddie_mercury 8y ago
Jenkins is "heavyweight" but Postgres isn't?
- guywhocodes 8y ago
  In what way is it not? I'm really curious actually
  - deleted 8y ago
    [deleted]
- kalleboo 8y ago
  Well if Postgres is already in your stack, it's no extra maintenance.
- sanderjd 8y ago
  When I was running both at the same time about five years ago, yeah, I personally found Jenkins to be more operationally expensive than a small postgresql instance. It required more memory, more CPU, and more time to set up and keep running as desired. Maybe things have improved or maybe it's just a YMMV thing, but yeah, I would have said the same thing as your parent without thinking twice. Edit to add: oh yeah, I just read another comment that reminded me that it also ate up lots of disk space as well.
- thatjsguy 8y ago
  As someone who has administrated both, I’d rather manage 10 Postgres instances than one Jenkins box. No question. Edit: I should expound. Jenkins seems like it has a lot of clunky moving parts. It all works, and I’d rather use it than anything else, but it’s kind of like IKEA furniture: you use it because you have to, not necessarily because you want to. It’s also incredibly difficult to automate. I can configure Postgres with a config file or two and easily use Ansible to get the exact same instance every time. Jenkins has to be dragged into Automation Alley kicking and screaming. I partially blame this on the fact that Jenkins has nontrivial amounts of configuration that’s done via GUI. I approach a long-running Jenkins instance with the same fear and dread I approach a Windows box that hasn’t been restarted in six months. I.e. the box is now a snowflake and trying to make it reproducible and automated is going to be a bad time. I could go on, but as a devops critter, Postgres wins every time.
  - peterwwillis 8y ago
    Jenkins is probably 100 times easier to deploy, configure, operate, and scale than Postgres+whatever, which is a horrifying statement, but still true. If what you have to do is schedule and run arbitrary jobs on any kind of machine, it's no contest. They aren't even related. Postgres is a relational database, and Jenkins is a single Java process that stores flat files on local disk, connects to remote nodes with SSH, and has a thousand plugins. It's like comparing a missile with an airplane. One gets where it needs to go faster and more efficiently, and the other one transports people.
    - setquk 8y ago
      Jenkins is one of those things that looks good until you trip over the numerous bugs in it. Such as the slightly terribly leaky implementation that hoses itself once a week, the severely million inodes it decides to gobble up which makes backups a chore etc. Sure it works but from an admin perspective it’s horrid.
      - peterwwillis 8y ago
        Is this really more horrid than having to hire a DBA to maintain, operate and upgrade the cron replacement? I can write a script to identify and remediate bugs in apps. I can't so easily write a script to diagnose badly used database apps and get the developers to rewrite their queries to not blow up the database servers. It's much easier to set up, lock down, and maintain Jenkins than Postgres+whatever, imo. But I guess everything can go to hell depending on how it's used.
tbrock 8y ago
Use an ECS scheduled task or the equivalent in Kubernetes: problem solved. Docker containers run using a task scheduler are perfect for this.
JdeBP 8y ago
That is hardly "all the alternatives". Some of the alternatives to the Vixie cron family (https://news.ycombinator.com/item?id=17005677 https://news.ycombinator.com/item?id=17005677) are: * Uwe Ohse's uschedule, http://jdebp.uk./Softwares/uschedule/ http://jdebp.uk./Softwares/uschedule/ https://ohse.de/uwe/uschedule.html https://ohse.de/uwe/uschedule.html * Bruce Guenter's bcron, http://untroubled.org./bcron/ http://untroubled.org./bcron/ * GNU mcron, https://news.ycombinator.com/item?id=17002098 https://news.ycombinator.com/item?id=17002098 * Thibault Godouet's fcron, http://fcron.free.fr/ http://fcron.free.fr/ * Matt Dillon's dcron, http://www.jimpryor.net/linux/dcron.html http://www.jimpryor.net/linux/dcron.html Other toolsets include Paul Jarc's runwhen (http://code.dogmap.org/runwhen/ http://code.dogmap.org/runwhen/) which is designed for first-class services individually scheduling themselves.
- Cieplak 8y ago
  Thanks for sharing these. In my case, I needed a system capable of sending payments at precise intervals, ensuring that a single payment was sent exactly once, despite the job being shared by a group of servers (for redundancy and high availability). Postgres provides the central locking, so tasks can be handed out to the pool of workers, while also supporting failover by streaming to a warm-standby replica.
- atombender 8y ago
  To be fair, all of these are node-local, not distributed, which is what the parent was talking about. (Some of the ones you mention are also very old and unmaintained.) For a distributed cron, you need a more sophisticated, transactional system that can deal with node failures. There are many cron replacements, but they generally don't tackle the main problems with the original cron, such as the inability to retry, queue, prevent overlapping jobs, enforce timeouts, report errors and metrics in a machine-readable way (spewing out emails is not a good solution), etc.
  - mmt 8y ago
    I think what you're describing may be a batch sceduling system, such as PBS or LFS. Those seem to be rarer, since the legitimate use case for a distributed system (what used to be called "grid computing") were rare. Nowadays, I assume everyone just uses Yarn on Hadoop, when they think scaling "up" ends at a mid-range 2U server. I don't know how good its actual time-of-day/calendar based scheduling is, though.

freddie_mercury 8y ago

Jenkins is "heavyweight" but Postgres isn't?

guywhocodes 8y ago
In what way is it not? I'm really curious actually
- deleted 8y ago
  [deleted]
kalleboo 8y ago
Well if Postgres is already in your stack, it's no extra maintenance.
sanderjd 8y ago
When I was running both at the same time about five years ago, yeah, I personally found Jenkins to be more operationally expensive than a small postgresql instance. It required more memory, more CPU, and more time to set up and keep running as desired. Maybe things have improved or maybe it's just a YMMV thing, but yeah, I would have said the same thing as your parent without thinking twice. Edit to add: oh yeah, I just read another comment that reminded me that it also ate up lots of disk space as well.
thatjsguy 8y ago
As someone who has administrated both, I’d rather manage 10 Postgres instances than one Jenkins box. No question. Edit: I should expound. Jenkins seems like it has a lot of clunky moving parts. It all works, and I’d rather use it than anything else, but it’s kind of like IKEA furniture: you use it because you have to, not necessarily because you want to. It’s also incredibly difficult to automate. I can configure Postgres with a config file or two and easily use Ansible to get the exact same instance every time. Jenkins has to be dragged into Automation Alley kicking and screaming. I partially blame this on the fact that Jenkins has nontrivial amounts of configuration that’s done via GUI. I approach a long-running Jenkins instance with the same fear and dread I approach a Windows box that hasn’t been restarted in six months. I.e. the box is now a snowflake and trying to make it reproducible and automated is going to be a bad time. I could go on, but as a devops critter, Postgres wins every time.
- peterwwillis 8y ago
  Jenkins is probably 100 times easier to deploy, configure, operate, and scale than Postgres+whatever, which is a horrifying statement, but still true. If what you have to do is schedule and run arbitrary jobs on any kind of machine, it's no contest. They aren't even related. Postgres is a relational database, and Jenkins is a single Java process that stores flat files on local disk, connects to remote nodes with SSH, and has a thousand plugins. It's like comparing a missile with an airplane. One gets where it needs to go faster and more efficiently, and the other one transports people.
  - setquk 8y ago
    Jenkins is one of those things that looks good until you trip over the numerous bugs in it. Such as the slightly terribly leaky implementation that hoses itself once a week, the severely million inodes it decides to gobble up which makes backups a chore etc. Sure it works but from an admin perspective it’s horrid.
    - peterwwillis 8y ago
      Is this really more horrid than having to hire a DBA to maintain, operate and upgrade the cron replacement? I can write a script to identify and remediate bugs in apps. I can't so easily write a script to diagnose badly used database apps and get the developers to rewrite their queries to not blow up the database servers. It's much easier to set up, lock down, and maintain Jenkins than Postgres+whatever, imo. But I guess everything can go to hell depending on how it's used.

tbrock 8y ago

Use an ECS scheduled task or the equivalent in Kubernetes: problem solved. Docker containers run using a task scheduler are perfect for this.

JdeBP 8y ago

That is hardly "all the alternatives". Some of the alternatives to the Vixie cron family (https://news.ycombinator.com/item?id=17005677 https://news.ycombinator.com/item?id=17005677) are: * Uwe Ohse's uschedule, http://jdebp.uk./Softwares/uschedule/ http://jdebp.uk./Softwares/uschedule/ https://ohse.de/uwe/uschedule.html https://ohse.de/uwe/uschedule.html * Bruce Guenter's bcron, http://untroubled.org./bcron/ http://untroubled.org./bcron/ * GNU mcron, https://news.ycombinator.com/item?id=17002098 https://news.ycombinator.com/item?id=17002098 * Thibault Godouet's fcron, http://fcron.free.fr/ http://fcron.free.fr/ * Matt Dillon's dcron, http://www.jimpryor.net/linux/dcron.html http://www.jimpryor.net/linux/dcron.html Other toolsets include Paul Jarc's runwhen (http://code.dogmap.org/runwhen/ http://code.dogmap.org/runwhen/) which is designed for first-class services individually scheduling themselves.

Cieplak 8y ago
Thanks for sharing these. In my case, I needed a system capable of sending payments at precise intervals, ensuring that a single payment was sent exactly once, despite the job being shared by a group of servers (for redundancy and high availability). Postgres provides the central locking, so tasks can be handed out to the pool of workers, while also supporting failover by streaming to a warm-standby replica.
atombender 8y ago
To be fair, all of these are node-local, not distributed, which is what the parent was talking about. (Some of the ones you mention are also very old and unmaintained.) For a distributed cron, you need a more sophisticated, transactional system that can deal with node failures. There are many cron replacements, but they generally don't tackle the main problems with the original cron, such as the inability to retry, queue, prevent overlapping jobs, enforce timeouts, report errors and metrics in a machine-readable way (spewing out emails is not a good solution), etc.
- mmt 8y ago
  I think what you're describing may be a batch sceduling system, such as PBS or LFS. Those seem to be rarer, since the legitimate use case for a distributed system (what used to be called "grid computing") were rare. Nowadays, I assume everyone just uses Yarn on Hadoop, when they think scaling "up" ends at a mid-range 2U server. I don't know how good its actual time-of-day/calendar based scheduling is, though.