3 ms·
To preface: I am not qualified to talk about this in the slightest. > Unfortunately, we discovered that a subset of services that were supposed to be on the hi
by AzzyHN 3y ago
To preface: I am not qualified to talk about this in the slightest.
> Unfortunately, we discovered that a subset of services that were supposed to be on the high availability cluster had dependencies on services exclusively running in PDX-04
What do you mean you discovered? How could you not know? Surely when you were setting this high availability cluster up years ago and migrating services over, you double checked that all crucial dependencies had also been moved, right? And surely, since you had been "implementing" this for four years now, you've TESTED what would happen if one of the three DCs went completely offline, right???
- gpm 3y ago> And surely, since you had been "implementing" this for four years now, you've TESTED what would happen if one of the three DCs went completely offline, right??? They discussed this. They had been running tests were they disabled the high availability cluster in any of (and of two of) the three DCs. That test didn't involve disabling the rest of the (non-HA) services from PDX-04 DC (oops).