3 ms·
It's necessary, but insufficient. You should have automated testing for each individual component. The problem is that by itself, automated testing won't give
by nostrademons 11y ago
It's necessary, but insufficient. You should have automated testing for each individual component. The problem is that by itself, automated testing won't give you a reliable distributed system. You need to assume that components will fail in ways that won't be caught by your tests, and design your architecture accordingly.
(Among other problems, automated testing won't catch a hardware failure, or someone tripping over the power cord, or a cosmic ray flipping a bit and corrupting memory.)
- byroot 11y ago> automated testing won't catch a hardware failure, or someone tripping over the power cord While it's never 100%, you can still test quite a lot of those conditions. We developed a TCP proxy[0] that allows us to test such cases: Datastore X being down / timing out / being excessively slow. [0] https://github.com/shopify/toxiproxy https://github.com/shopify/toxiproxy
- jacques_chester 11y agoNetflix approach this with their "chaos engineering" -- characterise fault behaviour by triggering known faults.