3 ms·
The best way to test a distributed system is to break it intentionally after you think you have sufficient monitoring to find all the problems. Then you'll kno
by jedberg 10y ago
The best way to test a distributed system is to break it intentionally after you think you have sufficient monitoring to find all the problems. Then you'll know if you have sufficient monitoring.
And if you have an outage where you didn't have sufficient monitoring, then you add it afterwards. But at least by breaking it intentionally you can at least be watching.
- timboio 10y ago+1 This is the entire idea behind Netflix's Failure Injection framework (http://techblog.netflix.com/2014/10/fit-failure-injection-testing.html http://techblog.netflix.com/2014/10/fit-failure-injection-te...) which was briefly alluded to in OP's link.
- joneholland 10y agoWhich makes sense considering jedburg works (worked?) for Netflix after he left Reddit.
- jedberg 10y agoWorked, but yes I was deeply involved in that project.