4 ms·

> Because there are soft costs to integrating a completely different platform into your environments. What happens when you try to pass a VM from an Intel syste

by cyphar 7y ago

> Because there are soft costs to integrating a completely different platform into your environments. What happens when you try to pass a VM from an Intel system over to an Epyc system? Unless they have the same instruction sets you can't pass them between different processors - meaning, you have to go in and manually find the greatest common denominator and disable the rest of the instructions that aren't mutually supported. That kind of thing. That shouldn't be a problem. They are both fundamentally the same architecture (amd64) and any CPU-specific features are already opportunistically handled by the vast majority of software because otherwise you wouldn't be able to run the same code on different versions of Intel's CPUs.

paulmd 7y ago
OSs are not most software and are not designed around the instruction set changing underneath them during normal operation. Why would they be? You can't physically swap out a processor while the system is booted and you can't swap a virtual processor either. It works fine if you shut everything down and reboot the system, but that is often undesirable. The whole point of the feature is that the VM can be migrated around different physical hardware without having to interrupt service. It just suddenly is running on a different host instance. But it has to be the same type of processor... or at least the same feature set. "Close" is not good enough, it needs to be a 1:1 match. You can manually disable features until you have found the lowest common denominator between the feature sets of the different processors. But obviously the more types of processors you have in your cluster, the more problematic this is. In very few clusters will you find servers of mixed types, you buy 10,000 of the same server and operate them as a "unit". You don't just add in servers after the fact, sometimes you don't even replace failed servers. And that hardware decision will have been made years ago, very often. The server market is hugely inertial, it's nothing like you putting together a build one evening and then going out and buying parts and putting it together.
- Nyandalized 7y ago
  >You can't physically swap out a processor while the system is booted and you can't swap a virtual processor either. I think you can do that on multisocket systems: https://www.kernel.org/doc/html/v4.14/core-api/cpu_hotplug.html https://www.kernel.org/doc/html/v4.14/core-api/cpu_hotplug.h...
  - Filligree 7y ago
    If the hardware supports it, yes, but only with basically the same type of processor.
TomVDB 7y ago
> That shouldn't be a problem. They are both fundamentally the same architecture (amd64) and any CPU-specific features are already opportunistically handled by the vast majority of software because otherwise you wouldn't be able to run the same code on different versions of Intel's CPUs. A very long time ago, I worked for a then very large company that sold servers. Plain standard 80486 based servers. My job was to drive around and drop off these servers for evaluation at prospective customers, who would compare them against 80486 offerings from a different vendor. Your argument about them all being fundamentally the same would be even stronger: it’s the same CPU. And yet, customers did not take chances and would go through the eval motions. Because their business relied on it. Now imagine that at a scale of thousands. Claiming “they are fundamentally the same” is not wrong, but you don’t care about the fundamentals only. You care about the whole picture and you don’t take chances.
- HelloNurse 7y ago
  Paying more for lower performance and higher TDP isn't a "chance". Very conservative corporate customers could wait a short time for good BIOS corrections and sufficient supply for all the parts (not only CPUs) they need before shopping for AMD servers, but they would be buying different hardware from the same established suppliers even if they went with Intel.
  - magashna 7y ago
    No one has ever been fired for buying Intel. Management often doesn't want to chance it on the underdog when they could keep everything the same
  - TomVDB 7y ago
    > Very conservative corporate customers could wait a short time for good BIOS corrections ... That sentence above is a contradiction in terms. A conservative corporate customers spends many months to do evaluations. There is no such thing as “a short time.”

by cyphar 7y ago

paulmd 7y ago

OSs are not most software and are not designed around the instruction set changing underneath them during normal operation. Why would they be? You can't physically swap out a processor while the system is booted and you can't swap a virtual processor either. It works fine if you shut everything down and reboot the system, but that is often undesirable. The whole point of the feature is that the VM can be migrated around different physical hardware without having to interrupt service. It just suddenly is running on a different host instance. But it has to be the same type of processor... or at least the same feature set. "Close" is not good enough, it needs to be a 1:1 match. You can manually disable features until you have found the lowest common denominator between the feature sets of the different processors. But obviously the more types of processors you have in your cluster, the more problematic this is. In very few clusters will you find servers of mixed types, you buy 10,000 of the same server and operate them as a "unit". You don't just add in servers after the fact, sometimes you don't even replace failed servers. And that hardware decision will have been made years ago, very often. The server market is hugely inertial, it's nothing like you putting together a build one evening and then going out and buying parts and putting it together.

Nyandalized 7y ago
>You can't physically swap out a processor while the system is booted and you can't swap a virtual processor either. I think you can do that on multisocket systems: https://www.kernel.org/doc/html/v4.14/core-api/cpu_hotplug.html https://www.kernel.org/doc/html/v4.14/core-api/cpu_hotplug.h...
- Filligree 7y ago
  If the hardware supports it, yes, but only with basically the same type of processor.

TomVDB 7y ago

> That shouldn't be a problem. They are both fundamentally the same architecture (amd64) and any CPU-specific features are already opportunistically handled by the vast majority of software because otherwise you wouldn't be able to run the same code on different versions of Intel's CPUs. A very long time ago, I worked for a then very large company that sold servers. Plain standard 80486 based servers. My job was to drive around and drop off these servers for evaluation at prospective customers, who would compare them against 80486 offerings from a different vendor. Your argument about them all being fundamentally the same would be even stronger: it’s the same CPU. And yet, customers did not take chances and would go through the eval motions. Because their business relied on it. Now imagine that at a scale of thousands. Claiming “they are fundamentally the same” is not wrong, but you don’t care about the fundamentals only. You care about the whole picture and you don’t take chances.

HelloNurse 7y ago
Paying more for lower performance and higher TDP isn't a "chance". Very conservative corporate customers could wait a short time for good BIOS corrections and sufficient supply for all the parts (not only CPUs) they need before shopping for AMD servers, but they would be buying different hardware from the same established suppliers even if they went with Intel.
- magashna 7y ago
  No one has ever been fired for buying Intel. Management often doesn't want to chance it on the underdog when they could keep everything the same
- TomVDB 7y ago
  > Very conservative corporate customers could wait a short time for good BIOS corrections ... That sentence above is a contradiction in terms. A conservative corporate customers spends many months to do evaluations. There is no such thing as “a short time.”