3 ms·
It's not that black and white. Where do you draw the line on what can and can't be a failure point? My cloud provider is probably an acceptable point. If every
by willsmith72 3y ago
It's not that black and white. Where do you draw the line on what can and can't be a failure point?
My cloud provider is probably an acceptable point. If every AWS region goes down I'm not going to have a spare cloud provider.
What about an auth provider? Do I need a backup there?
What about CI, do I need multiple CI systems?
3rd party search services, realtime messaging services, the list goes on.
For 1% of systems, you need backups for all of these (or to not use anything external). The other 99%, building backups for every one of these systems is a losing business strategy.
Some of them sure, but which those are will vary based on the context. It's not as simple as having a backup for "every other kind of remotely distributed dependency."
- GrinningFool 3y agoThat's all besides the point isn't it? Everything you mentioned can be replaced. Your source code can't.
- wongarsu 3y agoIt depends on the Recovery Time Objective we are talking about. For example at our company there are daily backups of both our dependencies and those used in the build process. If github had a prolonged outage or accidentally deleted all their data we could set up a gitea with all our code within hours, and get a replacement for Github Actions working within days. But that takes longer than the expected duration of this outage, and is a lot of work. It's not like we have a standby gitea we can just seamlessly switch to, so we are still hit by this outage. On the other hand for build dependencies we do have a standby mirror.
- Animats 3y ago> What about an auth provider? Do I need a backup there? Relying on Google for that would probably be a mistake. With cancelled projects and cancelled users, that's high risk.
- callalex 3y agoI draw that line at around 99.9% availability.
- BHSPitMonkey 3y agoAvailability is a measured outcome. How's it gonna help you quantify your various risks _before_ they become a problem so you can spend your mitigation time wisely?
- callalex 3y agoI was too brief, I meant I would add redundancy reactively where there was none there before if a service I’m using can’t consistently keep 4 9’s. You are correct that being proactive is more complicated.