5 ms·
How to run cron jobs in Postgres without extra infrastructure
- xnx 1y agoNo mention of pg_cron?
- etchalon 1y agoIt's what I expected to be talked about exclusively in the article based on the title.
- eddythompson80 1y agoapples and oranges? pg_cron is for pg specific cron tasks. You use pg_cron to truncate a table, compute pg views, values, aggregates, etc. Basically just running PG queries on a CRON schedule. pg_cron itself won't run an external script for you. Like you can't do SELECT cron.schedule('0/30 * * * *', $$ ./sendEmails.sh $$); you can use pg_cron to insert a job-row in a jobs table that you have some consumer that runs a `select * from jobs where status = 'pending' limit 1;`. Then you're on the hook to handle the pg updates for dispatching and handling updates, job status, etc. You could even call that implementation pg-boss if it's not taken.
- hoppp 1y agoThere is an HTTP extension for postgres, so it can trigger external serverless functions via http request
- cpursley 1y agoWhat’s the name of that?
- hoppp 1y agohttps://github.com/pramsey/pgsql-http https://github.com/pramsey/pgsql-http It works well with Supabase, I tried it, its decent but you should only use it for endpoints you trust because waiting for the request is blocking. If you want the requests to be async you need to use pg_background extension with it
- hyperman1 1y agoThe postgres COPY FROM PROGRAM will run external scripts, as the postgres user. Not necessarily a good architecture, of course. I did one day manage to fix a broken sshd with it by passing it su commands (rate that experience as 0 stars, would not recommend)
- SoftTalker 1y agoCron isn't an acronym; it's not normally written in all caps. Cron's name originates from Chronos, at least according to Wikipedia.
- tbrownaw 1y agoI can't check at the moment, but IIRC the output of `ps` on $employer's AIX boxes disagrees about it not being all-caps.
- deleted 1y ago[deleted]
- NeutralForest 1y agoTangential since it's not PG related but I'm more and more moving away from cron and I prefer using systemd timers (I'm on RHEL at work). I just find the interface to list and manager timers better and I can just handle everything like a systemd service anyways.
- jimis 1y agoWhat is the systemd equivalent for `service crond stop` and later `service crond start`? In other words, I want to disable all jobs for some time (for benchmarking) and then bring them back up.
- sherburt3 1y agoMaybe you could make a target unit file like “jobs.target” and in your timer unit files do “WantedBy=jobs.target”. Then you could do “systemctl start/stop jobs.target”
- r2_pilot 1y agoFirst, list and save the currently active timers: ```bash systemctl list-timers --state=active --no-legend | awk '{print $NF}' > /tmp/active_timers.txt ``` Stop all active timers: ```bash sudo systemctl stop $(cat /tmp/active_timers.txt) ``` Later, restart the previously active timers: ```bash sudo systemctl start $(cat /tmp/active_timers.txt) ```
- samtheprogram 1y agoI would try *.timer. If you’re in zsh, quote it.
- NeutralForest 1y agoLike the others said, you have to list them and save it somewhere, it could be better in that regard.
- zie 1y agoI have nothing against pg_boss[0] from the articel (I don't know anything about it), but there are plenty of queues and crons and schedulers for PG Some others: * https://github.com/LaunchPlatform/bq https://github.com/LaunchPlatform/bq * https://github.com/cybertec-postgresql/pg_timetable https://github.com/cybertec-postgresql/pg_timetable * https://github.com/pgmq/pgmq https://github.com/pgmq/pgmq * https://github.com/riverqueue/river https://github.com/riverqueue/river * https://github.com/oban-bg/oban https://github.com/oban-bg/oban * https://github.com/pgadmin-org/pgagent https://github.com/pgadmin-org/pgagent * https://github.com/citusdata/pg_cron https://github.com/citusdata/pg_cron etc. There are plenty of options to choose from. 0: https://github.com/timgit/pg-boss https://github.com/timgit/pg-boss
- TkTech 1y agoGonna toss my own hat in the ring there for the python+postgres ecosystem :) https://github.com/tktech/chancy https://github.com/tktech/chancy > As a rule of thumb, if you're processing less than 1000 jobs per day or your jobs are mostly lightweight operations (like sending emails or updating records), you can stick with this solution. This seems... excessively low? Chancy is on the heavier side and happily does many millions of jobs per day. Postgres has no issue with such low throughput, even on resource constrained systems (think a $5 vps). Maybe they meant 1000 per second?
- zie 1y agoI missed that. That does seem very small, 1k jobs/day is nothing. Chancy also looks pretty neat. Thanks for sharing!
- cpursley 1y agoAlso worth mentioning: https://www.pgflow.dev/ https://www.pgflow.dev/
- wewewedxfgdf 1y agoThere's many ways to skin this cat. Personally I invested all my knowledge and focus into systemd timers. No doubt you have your own ways that make sense for you.
- verdverm 1y agoThere's no systemd running in containers, so not an option in a lot of common scenarios
- sampullman 1y agoI haven't done it myself, but it seems possible with Podman or LXC containers. There's systemd-nspawn, too.
- hiAndrewQuinn 1y agoI like systemd when I have it; on the other end is the BusyBox cron implementation https://wiki.alpinelinux.org/wiki/Cron https://wiki.alpinelinux.org/wiki/Cron
- verdverm 1y agoI recently used PG-Boss to setup jobs to refresh auth tokens in the background. Very easy to use, would recommend taking a look. Docs are a bit minimal, but there's not that much to it either. (https://timgit.github.io/pg-boss/#/ https://timgit.github.io/pg-boss/#/) You don't need WASP for any of this, certainly not worth learning their custom DSL for it. Two of their points about how it makes it better are moot, setting queue names (one line of code) and type safety (you should be using TS already). I've not seen the value in their abstractions and indirection.
- lukasb 1y agoI can't be the only Next.js / neon user looking at this
- mitjam 1y agoKubernetes CronJobs are nice and if you are on K8s, already, it’s also without extra infrastructure.
- mati365 1y agoThis article seems to be written entirely by AI :/
- jbverschoor 1y agoCron/systemd/launchd is nice for machine-level tasks. If you want application or platform level tasks, you’re better off scheduling a task on which ever job queue you run. That could also be pg. That way you can have platform-wide unique tasks, probably better monitoring / tracing, etc.
- jackb4040 1y agoI have a node app that has one-off scheduled tasks. Between node-cron and real Linux cron, I went with real cron because node-cron just polls every second, which is extremely inefficient and I'm on a free tier. How does your library work in this regard? If my node server is down, will my scheduled tasks still execute? I notice you have a .start() method, what does that do? Is it polling periodically?
- xqzv 1y agoIt's polling using javascript timers: https://github.com/timgit/pg-boss/blob/master/src/attorney.js#L6 https://github.com/timgit/pg-boss/blob/master/src/attorney.j...
- OJFord 1y agoOr the aptly named pg_cron which is in RDS for example. TFA is just a marketing piece for Wasp, presumably to improve its SEO since 'postgres cron' more obviously gets you to pg_cron otherwise.