4 ms·

So how often do EC2 instances go down? Is it at hardware fail rate? or more often? Can I use this as a VPS replacement, and not have to worry about monitoring

So how often do EC2 instances go down? Is it at hardware fail rate? or more often? Can I use this as a VPS replacement, and not have to worry about monitoring and fast restoration? (of non-important projects).

mchadwick 16y ago
We've been running hundreds of instances on EC2 for a couple years, and have never seen one just "go down." However, we will get notifications of "degraded instances." When an instance is degraded, you have some window (generally a couple days) to move the services running on that instance to another one. Even at the aforementioned scale, this happens maybe once every three to four months. Can you use this as a VPS replacement? Probably. My guess is that your uptime will be no worse than some VPS provider. However, if you're storing information on the ephemeral storage, the onus is on you to get it to the new instance. I imagine that isn't generally the case on a VPS. You may be able to mitigate this by using EBS (required in the case of these micro instances), but I've only used EBS a handful of times, and am no expert on the subject. If I understand their layout for these micro instances, it would simply be a matter of spinning up a new instance and spinning down the degraded node.
mceachen 16y ago
I think you'd need a pretty large sample set before you came to any reasonable conclusion. I've been running ~10 nodes for AdGrok for the last 3 months, and we've already had one node fail (in that it wouldn't respond to any ec2 cli command to shut down or even terminate). That hardware failure rate is about what I'd expect if it was our own colo and our own machines. Stuff always breaks.
ary 16y ago
It's difficult to say. I've worked at two companies that have used EC2 for various purposes. One had an instance that was used for dev work get "corrupted" on three separate occasions (I couldn't get the specifics, but there were definitely I/O issues with the instance storage), and the other has been using the same production box for more than a year. The bottom line is that EC2 installations need to be designed as semi-permanent. My preferred strategy is similar to how Google talks about their hardware (when they do), that any one server can go down at any time, but the overall setup is resilient to failure.

mchadwick 16y ago

We've been running hundreds of instances on EC2 for a couple years, and have never seen one just "go down." However, we will get notifications of "degraded instances." When an instance is degraded, you have some window (generally a couple days) to move the services running on that instance to another one. Even at the aforementioned scale, this happens maybe once every three to four months. Can you use this as a VPS replacement? Probably. My guess is that your uptime will be no worse than some VPS provider. However, if you're storing information on the ephemeral storage, the onus is on you to get it to the new instance. I imagine that isn't generally the case on a VPS. You may be able to mitigate this by using EBS (required in the case of these micro instances), but I've only used EBS a handful of times, and am no expert on the subject. If I understand their layout for these micro instances, it would simply be a matter of spinning up a new instance and spinning down the degraded node.

mceachen 16y ago

I think you'd need a pretty large sample set before you came to any reasonable conclusion. I've been running ~10 nodes for AdGrok for the last 3 months, and we've already had one node fail (in that it wouldn't respond to any ec2 cli command to shut down or even terminate). That hardware failure rate is about what I'd expect if it was our own colo and our own machines. Stuff always breaks.

ary 16y ago

It's difficult to say. I've worked at two companies that have used EC2 for various purposes. One had an instance that was used for dev work get "corrupted" on three separate occasions (I couldn't get the specifics, but there were definitely I/O issues with the instance storage), and the other has been using the same production box for more than a year. The bottom line is that EC2 installations need to be designed as semi-permanent. My preferred strategy is similar to how Google talks about their hardware (when they do), that any one server can go down at any time, but the overall setup is resilient to failure.