3 ms·
something that i dont think gets talked about or addressed enough are partial failures. a whole instance dying is pretty much a solved problem and i think most
by tayo42 8y ago
something that i dont think gets talked about or addressed enough are partial failures. a whole instance dying is pretty much a solved problem and i think most people are aware of it and how to deal with it. if its not addressed people are just taking their chances at this point. a server being removed is an on and off kind of thing. something sporadic latency or corrupted data can cause some surprising and unpredictable issues.
- jedberg 8y agoChaos Monkey does that too. We used to have a separate monkey called Latency Monkey that induced network latency, because as you aptly point out, detecting if something is down is a lot easier than detecting if it is slow or intermittently down. Netflix has since incorporated those failure modes into chaos monkey: https://github.com/Netflix/SimianArmy/wiki/The-Chaos-Monkey-Army https://github.com/Netflix/SimianArmy/wiki/The-Chaos-Monkey-...