4 ms·

The odds that they would happen simultaneously if they’re completely unrelated seem astronomically small, certainly? Both are noted as being related to “ongoin

The odds that they would happen simultaneously if they’re completely unrelated seem astronomically small, certainly? Both are noted as being related to “ongoing migrations,” though AFAICT not related ones. I would bet there’s a human factor connection- e.g., the day before there was a big meeting where a higher-level management gave multiple ops teams go-ahead on their respective plans, resulting in a multiple potentially breaking changes occurring at the same time.

NoodleIncident 6y ago
I think it's as simple as a case of the Mondays; you wouldn't roll out a migration like that on a Friday or the weekend, and rolling it out on a Monday gives you the least chances of problem occurring on those dates.
- wolco2 6y ago
  A Monday rollout sounds horrible. Probably the worst day for a rollout.
  - alexchamberlain 6y ago
    Why is that?
  - caturopath 6y ago
    What about Saturday? Sunday? Friday?
  - skj 6y ago
    GCP rollouts happen over four days, and don't run on Friday. So, Monday it is! (there is wiggle room, exception granting, and grandfathering on this policy but it's true for many things)
speedgoose 6y ago
I think the likelyhoood of two incidents happening in the same period is not astronomically small, and is a variant of the birthday problem. It's a bit counter intuitive but if you have a few incidents during a year, the probability to have two incidents the same week is a lot higher than what you would expect. https://en.m.wikipedia.org/wiki/Birthday_problem https://en.m.wikipedia.org/wiki/Birthday_problem
- byecomputer 6y ago
  The birthday problem involves random people with unrelated birthdays. This is like two not-random siblings both calling in sick in the same week. The case of two large Google outrages where the whole service gets conked has much more potential to have a shared or related cause than two birthdays happening at once. You're right that it's not astronomically small, but the odds of them being related seem healthier than the odds of them being unrelated.

NoodleIncident 6y ago

I think it's as simple as a case of the Mondays; you wouldn't roll out a migration like that on a Friday or the weekend, and rolling it out on a Monday gives you the least chances of problem occurring on those dates.

wolco2 6y ago
A Monday rollout sounds horrible. Probably the worst day for a rollout.
- alexchamberlain 6y ago
  Why is that?
- caturopath 6y ago
  What about Saturday? Sunday? Friday?
- skj 6y ago
  GCP rollouts happen over four days, and don't run on Friday. So, Monday it is! (there is wiggle room, exception granting, and grandfathering on this policy but it's true for many things)

speedgoose 6y ago

I think the likelyhoood of two incidents happening in the same period is not astronomically small, and is a variant of the birthday problem. It's a bit counter intuitive but if you have a few incidents during a year, the probability to have two incidents the same week is a lot higher than what you would expect. https://en.m.wikipedia.org/wiki/Birthday_problem https://en.m.wikipedia.org/wiki/Birthday_problem

byecomputer 6y ago
The birthday problem involves random people with unrelated birthdays. This is like two not-random siblings both calling in sick in the same week. The case of two large Google outrages where the whole service gets conked has much more potential to have a shared or related cause than two birthdays happening at once. You're right that it's not astronomically small, but the odds of them being related seem healthier than the odds of them being unrelated.