10 ms·
The most interesting chaos experiments are the ones that show you gaps in your observability, when you know you've broken something and know you can't see it. T
by mentat 8y ago
The most interesting chaos experiments are the ones that show you gaps in your observability, when you know you've broken something and know you can't see it. The teams I've been on have learned that there are different sorts of observability that you need for different kinds of failure. Also, with complex systems, you need to learn how to operate your observability. Designing or testing resilience without being able to measure effectiveness is a time sink. Know that you can see failure first.