6 ms·

If your hosts have, say, dual 4-way CPUs, and you're giving your VMs 8 vCPUs, then a single VM can execute per clock cycle since a VM needs all of it's vCPUs ma

If your hosts have, say, dual 4-way CPUs, and you're giving your VMs 8 vCPUs, then a single VM can execute per clock cycle since a VM needs all of it's vCPUs made available to the guest OS. This seems wrong to me. We're talking about virtualization; technologically, it is absolutely feasibly to have only one physical CPU running a VM even if the VM sees multiple virtual CPUs. And it seems like having this capability in virtualization software from the very beginning is really a no-brainer. Evidence for your claim, please ;-)

X-Istence 13y ago
Sure, and if that one single CPU is emulating 8 vCPU's you haven't gained anything...
ckozlowski 13y ago
Certainly. Rather than retype the wheel though, I'm going to rely on some various posts: Gabe spells it out pretty well here: http://www.gabesvirtualworld.com/how-too-many-vcpus-can-negatively-affect-your-performance/ http://www.gabesvirtualworld.com/how-too-many-vcpus-can-nega... In a previous environment of mine, we ran dual socket, 4-way CPUs. VMs were configured with a mix of 1, 2, and 4 vCPU VMs. Our VMs appeared slow, especially on our 4-way systems. However, CPU utilization was low. More digging revealed that our co-stop values were high, meaning that the system couldn't schedule execution time effectively, meaning the VM had to sit in a READY state, which kept CPU utilization low. Our first fix was to rebalance our cluster, so that 1 and 2 vCPU VMs were relegated to their own set of hosts, and our 4 vCPU VMs executed on their own set. Instantly, out co-stop values dropped, and CPU utilization rates went up...they were now doing work! VMware briefly talks about the issue here: http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1005362 http://kb.vmware.com/selfservice/search.do?cmd=displayKC&doc... If you'd really like the nuts and bolts of it, then those can be found here: https://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf https://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CP... The VMware co-scheduler has improved over the years, but I still read (The "Mastering vSphere 5.5" book by Scott Lowe carries a warning on this as well) that carefully balancing vCPUs is a must in a VMware environment. (Again, I don't believe Linode uses VMware, so I can't say with any certainty that KVM or Xen exhibit this behavior.) So why can't we run 8 vCPUs on one physical one? Because while they're virtual to some extent, they're not completely abstracted. Anytime the hypervisor has to perform a translation between the guest OS and the host, a performance penalty is incured. So while the hypervisor may abstract scheduling, it reveals as much of the physical CPU to the guest VM as possible. Here's a little blurb from an older VMware manual explaining a bit of the difference: http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic=/com.vmware.vsphere.resourcemanagement.doc_41/managing_cpu_resources/c_understanding_cpu_virtualization.html http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic... CPU virtualization =/= emulation For this reason, (again, at least in a VMware environment) we can't give a VM more vCPUs than exist pCPUs to align them to. Hope this helps!
- mnordhoff 13y ago
  Fortunately, Linode runs Xen.

X-Istence 13y ago

Sure, and if that one single CPU is emulating 8 vCPU's you haven't gained anything...

ckozlowski 13y ago

Certainly. Rather than retype the wheel though, I'm going to rely on some various posts: Gabe spells it out pretty well here: http://www.gabesvirtualworld.com/how-too-many-vcpus-can-negatively-affect-your-performance/ http://www.gabesvirtualworld.com/how-too-many-vcpus-can-nega... In a previous environment of mine, we ran dual socket, 4-way CPUs. VMs were configured with a mix of 1, 2, and 4 vCPU VMs. Our VMs appeared slow, especially on our 4-way systems. However, CPU utilization was low. More digging revealed that our co-stop values were high, meaning that the system couldn't schedule execution time effectively, meaning the VM had to sit in a READY state, which kept CPU utilization low. Our first fix was to rebalance our cluster, so that 1 and 2 vCPU VMs were relegated to their own set of hosts, and our 4 vCPU VMs executed on their own set. Instantly, out co-stop values dropped, and CPU utilization rates went up...they were now doing work! VMware briefly talks about the issue here: http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1005362 http://kb.vmware.com/selfservice/search.do?cmd=displayKC&doc... If you'd really like the nuts and bolts of it, then those can be found here: https://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf https://www.vmware.com/files/pdf/techpaper/VMware-vSphere-CP... The VMware co-scheduler has improved over the years, but I still read (The "Mastering vSphere 5.5" book by Scott Lowe carries a warning on this as well) that carefully balancing vCPUs is a must in a VMware environment. (Again, I don't believe Linode uses VMware, so I can't say with any certainty that KVM or Xen exhibit this behavior.) So why can't we run 8 vCPUs on one physical one? Because while they're virtual to some extent, they're not completely abstracted. Anytime the hypervisor has to perform a translation between the guest OS and the host, a performance penalty is incured. So while the hypervisor may abstract scheduling, it reveals as much of the physical CPU to the guest VM as possible. Here's a little blurb from an older VMware manual explaining a bit of the difference: http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic=/com.vmware.vsphere.resourcemanagement.doc_41/managing_cpu_resources/c_understanding_cpu_virtualization.html http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic... CPU virtualization =/= emulation For this reason, (again, at least in a VMware environment) we can't give a VM more vCPUs than exist pCPUs to align them to. Hope this helps!

mnordhoff 13y ago
Fortunately, Linode runs Xen.