2 ms·
I use "cloud" services, but I ensure that my systems continue even if they fail. If AWS is down, maybe I can't do some analysis on historic logs until it comes
by iso1631 5y ago
I use "cloud" services, but I ensure that my systems continue even if they fail. If AWS is down, maybe I can't do some analysis on historic logs until it comes back, that's a known failure
I look at the components of a system and think "what happens if this is turned off // breaks in an unusual way // goes slow", and ensure that the predicted effects are known and acceptable based on the likelihood of failure.
That's the same whether it's an AWS managed DNS service, storage bucket, or a raspberry pi on my desk. As a systems engineer I know what that component does, what happens when it doesn't "do", and ensure the business knows how to work around it when it breaks.
If your business can't cope with an AWS outage (even if it's not as efficient) then you've got problems.
Plan for failure, and it doesn't take you by surprise.
- hellojesus 5y agoThis reminds me of something a volunteer at our campus observatory told me once while I was in college. I don't recall what we were discussing, but their comment stands out to me today: "Don't engineer things to work. Engineer things not to fail." Often at work I see code implementations that "work" in that they usually work but can fail. I'm not a great engineer by any means (I'm actually an economist that stumbled into code), but I believe that one of the reasons I've been able to gain a good reputation at work is specifically because I design with that principle in mind.