2 ms·
I'm not sure which design issue you're referring to; we absolutely did consider this scenario and the system behaved essentially as designed in this regard (in
by bcantrill 12y ago
I'm not sure which design issue you're referring to; we absolutely did consider this scenario and the system behaved essentially as designed in this regard (in particular, the contingency that we can determine a compute node's platform image while the control plane was impaired was critical for our relatively quick recovery). The only real issue with the recovery of compute nodes was the CPU throttling that we encountered as the entire fleet rebooted simultaneously, but that was quickly discovered and remediated (and would have only resulted in a longer recovery time, not an inability to recover).
Edit: the comment that I was replying to was apparently deleted. I won't attribute it, but here is the text I was replying to:
I'm generally the last person to finger point but wow! You guys didn't think about and test what happens when a lot of your servers reboot simultaneously? That happens any time you lose power! This exact issue has been raised and discussed every single time I've deployed any substantial number of PXE booting machines. There are actually a number of issues with relying on PXE booting and this is just one of them. This design make is far worse than the mistyped command.