3 ms·
This is fascinating. I am curious how sites like this (and others such as Pingdom, 'up time' checkers, etc) handle scheduled tasks that have to run at high freq
by rdoherty 4y ago
This is fascinating. I am curious how sites like this (and others such as Pingdom, 'up time' checkers, etc) handle scheduled tasks that have to run at high frequencies? Cron on one machine? Celery beat?
- cuu508 4y agoHealthchecks runs a loop of 10 send any due notifications 20 SLEEP 2 30 GOTO 10 The actual loop [1] is of course a little more complicated, and is being run concurrently on several machines. [1] https://github.com/healthchecks/healthchecks/blob/09a99d3e9c8329d20505e058a6514ba94cb17bb7/hc/api/management/commands/sendalerts.py#L151 https://github.com/healthchecks/healthchecks/blob/09a99d3e9c...
- adrienjarthon 4y agoIn my case (updown.io), I use a "scheduler" process (one per machine) which runs the following loop: every 5 seconds it fetches the checks that needs to execute in the next 5 seconds, ordered by time of run ascending. Then it iterates through them one by one, wait with a small sleep until the precise running time, and then push the job into a background job queue (sidekiq / redis in my case). This allows for a very precise timing of execution even if there is hundreds jobs per seconds, and a good distribution over time (instead of firing 100 jobs every seconds in spikes, it can schedule one every 10ms for example)