4 ms·
Automatic failover is like data backups, if not regularly tested it's like it doesn't exist.
by karmakaze 6y ago
Automatic failover is like data backups, if not regularly tested it's like it doesn't exist.
- vmception 6y agoThats a bit extreme in my experience, I would say its just closer to luck
- rwbhn 6y agoI would be very curious to hear about your favorable experience with untested failover systems. My last job we tested our active/standby failover like clockwork every 3 months. It served us very well.
- vmception 6y agoMy experience is that they worked when needed and it was a sigh of relief, and a little pat on the back that we perceived reality correctly when we had the foresight to set them up.
- ncmncm 6y agoAn untested failover system is indistinguishable from a box burning power and doing nothing. On a failure, it might work. But there might not be a failure. Same difference.
- karmakaze 6y agoIt's sort-of a saying in the industry that can be applied to many things. The biggest problem is the people who might know how to make it work are no longer there or have forgotten. If it's automated there's a chance it might work, but if it's slightly off then it's even harder to comprehend and adjust. I've seen backups and failovers not work so many times that it's an amazing surprise when one actually works--usually after laborious manual off-script intervention and invention. I was being somewhat kind to failovers, I have seen smallish backups work on several occasions. An untested disaster recovery is the worst. Edit: think of an unexercised procedure as not "It works for me" but rather "it worked for me once."
- renewiltord 6y agoFailover sounds like the sort of thing you just do all the time. i.e. at the extreme, you treat failover like loadbalancing.
- ptr 6y agoThese kinds of systems typically don’t use load balancing, they’re provisioned for the expected max capacity needed.
- solarkraft 6y agoIt seems like for true redundancy one should have multiple failover devices with regular actual switching between them.
- fomine3 6y agoI don't think failover won't work in every failure scenario, but won't work in this particular scenario. We can't test all possible failure situation regularly.