3 ms·
Testing the backups is harder then you think... It's not like you are going to double up your entire server fleet just to see if you can restore everything from
by z3t4 5y ago
Testing the backups is harder then you think... It's not like you are going to double up your entire server fleet just to see if you can restore everything from backup. You maybe test restoration of one or two servers and then assume the rest will also work. And you probably have some redundancy so that the data is saved on different machines plus a backup plus an additional backup. Then if things goes down, which is very, very unlikely, like a coordinated nuclear attack on several data-centers, but of course can happen, you assume you will figure things out... And then someone will ask: so what happens if your team also die in the nuclear attack, then you add one zero to the backup price/cost estimation.
- dougpa 5y agoFinding something reasonable between "we're at risk for a nuclear attack" (a black swan event) and "we're at risk for having our root credentials exfiltrated" (a daily occurrence) is not hard. I feel like these kinds of dramatizing hyperboles are nowhere near the nuts & bolts of the situation. And honestly, why not double up your entire server fleet for a temporary build-from-scratch rehearsal? Many shops could quintuple their infrastructural costs and still sit far above being in red. Most software enterprises in the modern day don't reap economic value as a function of how well they can convert hardware resources, but human resources. Optimizing on infra costs is not a main priority for any shop I've seen. I imagine not even for an IaaS provider these days.