5 ms·

>Visualization overhead for networking is too high for the cost. Do you mean virtualization? If so, I recommend looking into testing this with SR-IOV based NI

• mmt 8y ago
  With potentially substantial engineering effort, including needing to hire someone with a relatively rare expertise, they could [1] eliminate that specific downside of virtualization. However, there are remaining overhead/downsides, and virtualization may be a solution looking for a problem in their environment. [1] Also, presumably, dependent on specific NIC hardware, but I expect they're already using something compatible. It's merely another constraint.
  • gregdunn 8y ago
    By no means do you need DPDK to basically eliminate the latency difference - I just wanted to point out how low latency can go in general. A vm using SR-IOV with ixgbevf on good ol' Intel 82599 from 7 years ago will not have a latency difference noticeable to the overwhelming majority of use cases vs. bare metal.
    • mmt 8y ago
      > By no means do you need DPDK to basically eliminate the latency difference I didn't mean to imply that was your argument. > I just wanted to point out how low latency can go in general. Rather, I meant that "can" isn't the same as "does", absent exceptional circumstances. > A vm using SR-IOV Whether this qualifies as exceptional is, of course, arguable, but I'm arguing that it is. I could understand the point that it doesn't have to be, but, to be actually convinced, I'd want to see evidence that it's well understood and well implemented enough that neither rare expertise, substantial engineering effort, nor constrained configuration (hardware or software) would be required to take advantage of it. I'd expect most technically-minded decision makers to think similarly.
      • gregdunn 8y ago
        >Whether this qualifies as exceptional is, of course, arguable, but I'm arguing that it is. I could understand the point that it doesn't have to be, but, to be actually convinced, I'd want to see evidence that it's well understood and well implemented enough that neither rare expertise, substantial engineering effort, nor constrained configuration (hardware or software) would be required to take advantage of it. I'd expect most technically-minded decision makers to think similarly. Oh. My apologies for misunderstanding your point. SR-IOV is available on basically any and all server grade NICs, and is quite simple to use. With Azure and AWS it's basically just making sure you have the proper driver installed (gotten for free on basically all modern kernels) and flipping a command switch. If you're rolling your own virtualization stack, it's generally about as simple as any other task for that stack. With vSphere it takes a matter of seconds: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.networking.doc/GUID-EE03DC6F-32CA-42EF-98FC-12FDE06C0BE0.html https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsp... Similarly easy for XenServer: https://support.citrix.com/article/CTX126624 https://support.citrix.com/article/CTX126624 A little bit more work with the common KVM management options, but still a very simple task as far as Linux sysadmin tasks go: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_host_configuration_and_guest_installation_guide/sect-virtualization_host_configuration_and_guest_installation_guide-sr_iov-how_sr_iov_libvirt_works https://access.redhat.com/documentation/en-us/red_hat_enterp... OpenStack is a bit more complicated, but frankly, less complicated than plenty other tasks in OpenStack: https://docs.openstack.org/mitaka/networking-guide/config-sriov.html https://docs.openstack.org/mitaka/networking-guide/config-sr... All of the real setup work has to be done at the hypervisor level, but you're primarily just doing two things: Creating VFs, and assigning them to VMs. The driver does all of the rest of the hard work. I would argue that any Linux or vSphere admin with any real amount of experience should be able to read any of the documentation I linked and be able to confidently work through it in an hour or two. For the guest, just making sure the driver is installed should be all that's required. For ixgbevf, the ubiquitous commercial option, it's been in-tree for the Linux kernel for at least half a decade. Once VFs are created and assigned, it largely "Just Works". The only real caveat I know of is that seamless live migration of the guest is no longer an option, because now all of the network virtualization is handled in the hardware instead of the hypervisor.
• fulafel 8y ago
  Probably the biggest jitter comed from your VPS getting preempted by other customers due to oversubscribing (cpu or network), not the relatively benign and fixed amount of packet processing overhead from virtualization itself.
  • gregdunn 8y ago
    For sure! But that's a matter of the provider's placement decisions, and not inherent to virtualization.