We manage hundreds of servers and have been migrating more and more to spot instances in order to save on costs. It was only a portion of our infrastructure, b

We manage hundreds of servers and have been migrating more and more to spot instances in order to save on costs. It was only a portion of our infrastructure, but it was just enough to cause us grief. Given the volume of servers and the velocity with which our software has been changing, I'll readily admit it's been a challenge keeping ahead of everything and we definitely went a little too far with the spot instances in a few places, but we just haven't seen this failure mode before so we got overconfident.