9 ms·

If you are ever building a platform and have control over everything, one thing that can make problems like this easier to find is to not use regular intervals

If you are ever building a platform and have control over everything, one thing that can make problems like this easier to find is to not use regular intervals like 5/15/30/60 minutes everywhere. At some point you'll have a weird problem, or a load spike that shows up at regular intervals. If all of your intervals are 5/15/30 minutes, you will have 2 things running every 15 minutes and 3 things running every 30 minutes, you won't necessarily know which one causes the issue. If you use (co)prime numbers, say, 5/7/11/13/17/19 as intervals: One, you won't have a thundering herd of tasks all running at the exact same time every few minutes, and two, when someone notices a weird issue that happens every 17 minutes, you will know exactly what the cause is.

cozzyd 2y ago
wouldn't it have made it harder to find in this case if the DCHP lease time was 34 minutes?
- eszed 2y ago
  If it's in your control use co-prime scheduling. If it's not in your control, um... Hope they didn't use co-primes? Or a multiple of your co-primes? Er, yeah. I see the justification for doing it, but it's not exactly a cure for complexity. It'll work up until everyone else catches up with your weird trick, and then there might be may be more collisions than there were before. Edit: Yeah, I guess GP said "if you control everything". Still, how often do you actually control everything? Or how long can you control everything? Everything (heh) sufficiently complex to worry about this talks with other systems at some point, right?
  - justinsaccount 2y ago
    I think you underestimate how badly people mess this up now, with piles and piles of cron jobs and timers all configured to run at 1/5/15/60 minute intervals. If other things did this there would not be "more collisions", since the status quo is multiples of 5 minute intervals everywhere.
    - eszed 2y ago
      Fair point. I'll definitely consider co-primes the next time I deploy a system with complicated scheduling.
      - justinsaccount 2y ago
        for what it's worth, it also comes in super handy when trying to troubleshoot existing systems. Running a complicated system that has weird latency spikes sometimes at a 60s interval and the config file has like: - timeout: 1m - expireInterval: 1m - updateInterval: 5m - etc The first thing I do is go in and change them all to - timeout: 59s - expireInterval: 61s - updateInterval: 307s - etc and then see what the new "problem interval" is. It's either gone, because the spike was from contention from all 3 jobs running every 5 minutes, or the new problem occurs now at 61s intervals.
- justinsaccount 2y ago
  Why?

cozzyd 2y ago

wouldn't it have made it harder to find in this case if the DCHP lease time was 34 minutes?

eszed 2y ago
If it's in your control use co-prime scheduling. If it's not in your control, um... Hope they didn't use co-primes? Or a multiple of your co-primes? Er, yeah. I see the justification for doing it, but it's not exactly a cure for complexity. It'll work up until everyone else catches up with your weird trick, and then there might be may be more collisions than there were before. Edit: Yeah, I guess GP said "if you control everything". Still, how often do you actually control everything? Or how long can you control everything? Everything (heh) sufficiently complex to worry about this talks with other systems at some point, right?
- justinsaccount 2y ago
  I think you underestimate how badly people mess this up now, with piles and piles of cron jobs and timers all configured to run at 1/5/15/60 minute intervals. If other things did this there would not be "more collisions", since the status quo is multiples of 5 minute intervals everywhere.
  - eszed 2y ago
    Fair point. I'll definitely consider co-primes the next time I deploy a system with complicated scheduling.
    - justinsaccount 2y ago
      for what it's worth, it also comes in super handy when trying to troubleshoot existing systems. Running a complicated system that has weird latency spikes sometimes at a 60s interval and the config file has like: - timeout: 1m - expireInterval: 1m - updateInterval: 5m - etc The first thing I do is go in and change them all to - timeout: 59s - expireInterval: 61s - updateInterval: 307s - etc and then see what the new "problem interval" is. It's either gone, because the spike was from contention from all 3 jobs running every 5 minutes, or the new problem occurs now at 61s intervals.
justinsaccount 2y ago
Why?