5 ms·
Something that often goes ignored is that in high availability situations like that, humans are the biggest risk by a huge margin. Computers (almost always) don
by cheeze 7y ago
Something that often goes ignored is that in high availability situations like that, humans are the biggest risk by a huge margin. Computers (almost always) don't accidentally wipe out the prod database thinking it was beta, that's usually a human that made a simple mistake.
Dumping money into IaaS while neglecting to recognize that high availability requires extreme quality of both engineering, as well as testing/qa is a super common mistake that's being made these days. This means slowing down, which most upper management doesn't want to do.
Things like executing DR plans regularly to test that they wofk are extremely costly to the business,and are one of those "IT" things that you hopefully never have to actually do in production. But without things like that, what's the point of 3xing the cost of your fleet and adding complexity (another great vector to cause failure) for "availability"?
I think the answer often comes down tok the fact that people are hard to hire. Computers are easy to provision.
- FigmentEngine 7y agoYes, humans are really important in these situations, thats why "Operational Excellence" is the first pillar of AWS Well-Architected - you can only get so far in terms of reliability and security if you don't consider people and process. https://d1.awsstatic.com/whitepapers/architecture/AWS-Operational-Excellence-Pillar.pdf https://d1.awsstatic.com/whitepapers/architecture/AWS-Operat... (I work on the AWS Well-Architected team)
- crankylinuxuser 7y ago(erased)