4 ms·

That is a very different thing than 'changing under you', which implies that a running instance is being shuttled around to different underlying hardware, which

That is a very different thing than 'changing under you', which implies that a running instance is being shuttled around to different underlying hardware, which though I think is possible, isn't what is being described in TFA. We don't even know that the case you are talking about, of stopping, then starting an instance leads to it living on different hardware types, though I would suspect that is true. TFA is saying that when you boot NEW instances you sometimes get subtly different hardware, which when deploying binary code can lead to issues. That does seem like a valid EC2 bug, but also not one that most of us will ever run into unless you are doing cool massive dynamic scaling like they are.

vacri 14y ago
I don't know if AWS does it, but I remember going to a VMWare-sponsored industry day several years ago, and they described that for enterprise systems, the hardware was pooled and an abstraction layer sat between them and the guests. Guests could be migrated between machines while they were running and active. It allowed a system where in quiet times you could move your guest load to only part of your hardware group and put the rest into low-power mode (to save on power, it seems). The best part of that day was a guy from the government who gave a great talk on his experiences virtualising from running separate physical servers. While he was very much in favour of it, he mentioned a few drawbacks that the vendor talks obviously played down, but probably the best thing he said was "when doing something major with infrastructure like this, get your boss at the top on board early, because everyone above you will try to scuttle it to CYA if it goes wrong."
- dbarlett 14y ago
  VMWare calls it vMotion [1]. You define the host cluster and it handles the rest. Xen supports live migration [2] but EC2 does not. [1] http://www.vmware.com/products/datacenter-virtualization/vsphere/vmotion.html#glance http://www.vmware.com/products/datacenter-virtualization/vsp... [2] http://sysadmin.wikia.com/wiki/Live_migration_xen http://sysadmin.wikia.com/wiki/Live_migration_xen
  - spydum 14y ago
    Fwiw, vMotion checks the cpu supported flags compatibility before migrating. Pretty sure cpu must be all of the same family to be clustered together for this purpose.
    - jimwalsh 14y ago
      VMware EVC simplified the CPU requirements that you mention. It can be very restrictive still, or more lax depending on your requirements. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003212 http://kb.vmware.com/selfservice/microsites/search.do?langua... The original article just struck me as yet another company that doesn't know how to properly run applications in virtualized environments. Yes, if your hosting provider supports hot or cold migrations, you should be aware and develop accordingly. I do see this quite often, that still today, people are surprised that their VM is not always running on the same physical box at all times.
      - xyzzy123 14y ago
        I think you're being overly harsh. Picloud is a super-awesome abstraction layer for running Python over AWS. The problem they have is that they are trying to support arbitrary Python (which includes underlying libraries like LAPACK which depend on runtime CPU detection working correctly) and expecting code will work the same across identical instance types. I think normal people would consider this a fair assumption, and in the event that AWS's advertised CPU capabilities were not broken for some instances, it would hold. I don't think it's fair to say that they're ignorant of how virtualisation works. Disclaimer: I am a huge picloud fan and they have saved me a lot of time.
      - darkarmani 14y ago
        Did you even read the article? This has nothing to do with not running on the same physical box and more to do with some physical boxes are advertising incorrect instruction sets to the OS.
- usaar333 14y ago
  While EC2's hyppervisor, Xen, does support live migration, EC2 has no support for it. With that said, you can "migrate" EBS-backed instances by start/stopping them which (may) place them on different hardware Indeed, if you are large enough, you'll occasionally get emails from Amazon directing you to do just that due to scheduled maintenance on the physical machines.

vacri 14y ago

I don't know if AWS does it, but I remember going to a VMWare-sponsored industry day several years ago, and they described that for enterprise systems, the hardware was pooled and an abstraction layer sat between them and the guests. Guests could be migrated between machines while they were running and active. It allowed a system where in quiet times you could move your guest load to only part of your hardware group and put the rest into low-power mode (to save on power, it seems). The best part of that day was a guy from the government who gave a great talk on his experiences virtualising from running separate physical servers. While he was very much in favour of it, he mentioned a few drawbacks that the vendor talks obviously played down, but probably the best thing he said was "when doing something major with infrastructure like this, get your boss at the top on board early, because everyone above you will try to scuttle it to CYA if it goes wrong."

dbarlett 14y ago
VMWare calls it vMotion [1]. You define the host cluster and it handles the rest. Xen supports live migration [2] but EC2 does not. [1] http://www.vmware.com/products/datacenter-virtualization/vsphere/vmotion.html#glance http://www.vmware.com/products/datacenter-virtualization/vsp... [2] http://sysadmin.wikia.com/wiki/Live_migration_xen http://sysadmin.wikia.com/wiki/Live_migration_xen
- spydum 14y ago
  Fwiw, vMotion checks the cpu supported flags compatibility before migrating. Pretty sure cpu must be all of the same family to be clustered together for this purpose.
  - jimwalsh 14y ago
    VMware EVC simplified the CPU requirements that you mention. It can be very restrictive still, or more lax depending on your requirements. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003212 http://kb.vmware.com/selfservice/microsites/search.do?langua... The original article just struck me as yet another company that doesn't know how to properly run applications in virtualized environments. Yes, if your hosting provider supports hot or cold migrations, you should be aware and develop accordingly. I do see this quite often, that still today, people are surprised that their VM is not always running on the same physical box at all times.
    - xyzzy123 14y ago
      I think you're being overly harsh. Picloud is a super-awesome abstraction layer for running Python over AWS. The problem they have is that they are trying to support arbitrary Python (which includes underlying libraries like LAPACK which depend on runtime CPU detection working correctly) and expecting code will work the same across identical instance types. I think normal people would consider this a fair assumption, and in the event that AWS's advertised CPU capabilities were not broken for some instances, it would hold. I don't think it's fair to say that they're ignorant of how virtualisation works. Disclaimer: I am a huge picloud fan and they have saved me a lot of time.
    - darkarmani 14y ago
      Did you even read the article? This has nothing to do with not running on the same physical box and more to do with some physical boxes are advertising incorrect instruction sets to the OS.
usaar333 14y ago
While EC2's hyppervisor, Xen, does support live migration, EC2 has no support for it. With that said, you can "migrate" EBS-backed instances by start/stopping them which (may) place them on different hardware Indeed, if you are large enough, you'll occasionally get emails from Amazon directing you to do just that due to scheduled maintenance on the physical machines.