3 ms·

I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the

I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the wiser? Am I expecting too much?

ghshephard 13y ago
EC2 instances don't come with vmotion. It's up to the customer to detect a failed/retired node and restart on another EC2 instance. The first thing you discover when reading through the various options is that you need to treat ALL local storage like /tmp, subject to deletion at will. Keep your persistent storage on EBS/S3.
- arturhoo 13y ago
  And even if you do keep your important stuff on EBS, make sure you take snapshots on a frequent basis. We have received this email a couple of times: Your volume experienced a failure due to multiple failures of the underlying hardware components and we were unable to recover it. Although EBS volumes are designed for reliability, backed by multiple physical drives, we are still exposed to durability risks caused by concurrent hardware failures of multiple components, before our systems are able to restore the redundancy. We publish our durability expectations on the EBS detail page here (http://aws.amazon.com/ebs). Sincerely, EBS Support Fortunately, we had recent snapshots and it was a matter of (manually) spinning up a new instance from those. Edit: proper quotation
RSDnuyIsk9jMSWb 13y ago
Windows Azure actually does this. If the host your virtual machine is on for some reason fails or needs to be replaced your entire VM is migrated to another host. The migration process can take a few minutes but all your data is safe. My point being. On this topic AWS could learn from Microsoft on how to do cloud.
- SudoAlex 13y ago
  The one problem they have is that the majority of their instances include local storage, which would make migration impossible. So the best they can offer is a reboot so the server ends up on another host. They could potentially do this on their second generation (M3) instances, as well as micro instances if they wanted to. However I'd guess that these instances are just a small percentage of the overall servers used.
  - travem 13y ago
    > The one problem they have is that the majority of their instances include local storage, which would make migration impossible True for AWS. Using VMware vSphere then this could be done with a shared nothing migration which moves compute and storage (vMotion + storage vMotion combined).
    - justizin 13y ago
      Xen also does this, but AWS may still be using a very old fork of Xen 2 or 3. It's unfortunate that there doesn't seem to be a large-scale consumer for the XenServer platform now that Citrix open-sourced the entire thing.
cheeseprocedure 13y ago
While Xen should make live migrations technically possible, it would probably reduce EC2's provisioning flexibility and introduce undesirable complexity. Migrations would be restricted to hosts running specific releases of the hypervisor [1], and AWS's SDN systems would need to handle these changes in very-near-realtime. [1] wiki.xen.org/wiki/Xen_Version_Compatibility

I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the wiser? Am I expecting too much?

ghshephard 13y ago

EC2 instances don't come with vmotion. It's up to the customer to detect a failed/retired node and restart on another EC2 instance. The first thing you discover when reading through the various options is that you need to treat ALL local storage like /tmp, subject to deletion at will. Keep your persistent storage on EBS/S3.

arturhoo 13y ago
And even if you do keep your important stuff on EBS, make sure you take snapshots on a frequent basis. We have received this email a couple of times: Your volume experienced a failure due to multiple failures of the underlying hardware components and we were unable to recover it. Although EBS volumes are designed for reliability, backed by multiple physical drives, we are still exposed to durability risks caused by concurrent hardware failures of multiple components, before our systems are able to restore the redundancy. We publish our durability expectations on the EBS detail page here (http://aws.amazon.com/ebs). Sincerely, EBS Support Fortunately, we had recent snapshots and it was a matter of (manually) spinning up a new instance from those. Edit: proper quotation

RSDnuyIsk9jMSWb 13y ago

Windows Azure actually does this. If the host your virtual machine is on for some reason fails or needs to be replaced your entire VM is migrated to another host. The migration process can take a few minutes but all your data is safe. My point being. On this topic AWS could learn from Microsoft on how to do cloud.

SudoAlex 13y ago
The one problem they have is that the majority of their instances include local storage, which would make migration impossible. So the best they can offer is a reboot so the server ends up on another host. They could potentially do this on their second generation (M3) instances, as well as micro instances if they wanted to. However I'd guess that these instances are just a small percentage of the overall servers used.
- travem 13y ago
  > The one problem they have is that the majority of their instances include local storage, which would make migration impossible True for AWS. Using VMware vSphere then this could be done with a shared nothing migration which moves compute and storage (vMotion + storage vMotion combined).
  - justizin 13y ago
    Xen also does this, but AWS may still be using a very old fork of Xen 2 or 3. It's unfortunate that there doesn't seem to be a large-scale consumer for the XenServer platform now that Citrix open-sourced the entire thing.

cheeseprocedure 13y ago

While Xen should make live migrations technically possible, it would probably reduce EC2's provisioning flexibility and introduce undesirable complexity. Migrations would be restricted to hosts running specific releases of the hypervisor [1], and AWS's SDN systems would need to handle these changes in very-near-realtime. [1] wiki.xen.org/wiki/Xen_Version_Compatibility