3 ms·
Heh Heh Heh On one contract about a decade ago, the manager used to proactively screw around with server hardware (eg pull a hdd out of a raid set) when bored,
by justinclift 11y ago
Heh Heh Heh
On one contract about a decade ago, the manager used to proactively screw around with server hardware (eg pull a hdd out of a raid set) when bored, to ensure our recovery processes were always battle tested.
Didn't generally cause too many issues... until the one time he pulled a working hdd out of a raid5 set that already had a (real) failed drive, which we were already attempting to recover. It was a production database, but thankfully not a mission critical one :>.
From memory, he stopped testing things like that for a while afterwards. ;)
- cvwright 11y agoWow, a real-life chaos monkey! https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey
- justinclift 11y agoYeah, the similarity hasn't escaped me either. ;)