4 ms·

> How's much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? I've seen research that shows the GPU being 98-99

> How's much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? I've seen research that shows the GPU being 98-99% when using QEMU/KVM. I haven't used the system long enough to accurately quantify my personal experience. > Is it as simple as it seems? For me getting the GPU working was the easiest part. It worked on the first try. The hardest part was finding good docs (Arch wiki & VFIO blog), trying to understand their little ecosystem, and finding the right compatible parts (hint: besides the CPU, it's really easy). I'd never used QEMU/KVM before (I used VirtualBox previously) so that was a big learning curve. It seems like things have matured quite a bit since the VFIO blog wrote its articles, as a number of things had changed and become easier.

hbosch 11y ago
Does "getting the GPU to work" mean you can give your Windows VM exclusive rights to the ports on the back of that graphics card as well? That would be amazing. I'd love to, say, run a Hackintosh setup with Windows 10 in a VM that could fully utilize it's own GPU + external ports -- this way I could use a modern VR setup (Oculus or Vive) without having to use Windows full time!
- fpgaminer 11y ago
  In fact, that's the only way to do it. This VFIO/IOMMU setup virtualizes the PCI-E port itself. It routes the PCI-E port that the graphics card is plugged into to the VM. So the VM has exclusive and direct access to that PCI-E port. Therefore, the GPU, the ports on it, everything hooked up to that PCI-E port belongs to the VM. Most importantly, this also means the GPU's SMBus for temperature monitoring is also routed and owned by the VM. From the VM's perspective it isn't any different than a physical card. You can also route USB hubs to it the same way, sound cards, etc.
  - hbosch 11y ago
    I can't believe I've never heard of this before. I'm honestly astonished! Thanks for your reply, I can't wait to try this.
  - smellf 11y ago
    A coworker of mine has been wanting to do this for ages, and I've always wondered - what's the workflow like for getting the mouse and keyboard from the Linux screen to the Windows screen? For the past few years I've been running two desktops side by side (one with Windows and one with Linux), then use Synergy to get from one to the other painlessly. When I next upgrade, IOMMU might let me consolidate the number of PCs available, but only if it's relatively painless to get from one OS to the other.
    - fpgaminer 11y ago
      So, there are several ways to handle mouse+keyboard that I've seen. 1) Virtualized Mouse+Keyboard. If you've ever used a VM before, this is what you most likely are familiar with. You click within the VM's window, it captures your mouse and keyboard, and then everything is like normal within the VM. To release, you hit some special key combo and your mouse+keyboard is released back to the host. The VGA Passthrough setup that we're talking about doesn't normally support this method in QEMU/KVM, but you can hack it in. For me, it works as expected, except the mouse+keyboard randomly stop working for brief periods of time every so often. So... not great. 2) Synergy. I've seen this method mentioned a lot, so I guess it works. I haven't tried it. I've heard there are occasional issues, and sometimes Synergy will stop working so you need some kind of fallback method to get back into the Windows VM to get it working again. Probably this method plus method 1 above as a backup will work well. 3) USB passthrough. You could get a second mouse+keyboard, and then use USB filtering to pass them through to the VM. I haven't tried it; it probably works fine if you're okay with having two mouse+keyboard sets on your desk. 4) IOMMU USB passthrough. Similar to #3, but you use IOMMU to give an entire USB hub to the VM. This is likely to have the least number of issues, as it gives direct and exclusive control of a USB hub to the VM. Whether you can do it or not depends on the USB hub configuration on your mobo or whether you feel like adding a USB PCI/PCI-E card. 5) KVM + USB passthrough. Use a KVM and then USB passthrough of some kind (#3 or #4 above). 6) Steam In-Home Streaming. Steam In-Home Streaming can be hacked to share the entire desktop of the target machine. So you could use this to not only handle keyboard+mouse, but also video if you don't want a second monitor and/or don't want to use your existing monitor to switch HDMI/DisplayPort. Since the streaming will be done locally, latency will be very low, but this will burn some CPU/GPU resources. VNC is also an option, but will likely fallover for video games/watching videos/etc.

hbosch 11y ago

Does "getting the GPU to work" mean you can give your Windows VM exclusive rights to the ports on the back of that graphics card as well? That would be amazing. I'd love to, say, run a Hackintosh setup with Windows 10 in a VM that could fully utilize it's own GPU + external ports -- this way I could use a modern VR setup (Oculus or Vive) without having to use Windows full time!

fpgaminer 11y ago
In fact, that's the only way to do it. This VFIO/IOMMU setup virtualizes the PCI-E port itself. It routes the PCI-E port that the graphics card is plugged into to the VM. So the VM has exclusive and direct access to that PCI-E port. Therefore, the GPU, the ports on it, everything hooked up to that PCI-E port belongs to the VM. Most importantly, this also means the GPU's SMBus for temperature monitoring is also routed and owned by the VM. From the VM's perspective it isn't any different than a physical card. You can also route USB hubs to it the same way, sound cards, etc.
- hbosch 11y ago
  I can't believe I've never heard of this before. I'm honestly astonished! Thanks for your reply, I can't wait to try this.
- smellf 11y ago
  A coworker of mine has been wanting to do this for ages, and I've always wondered - what's the workflow like for getting the mouse and keyboard from the Linux screen to the Windows screen? For the past few years I've been running two desktops side by side (one with Windows and one with Linux), then use Synergy to get from one to the other painlessly. When I next upgrade, IOMMU might let me consolidate the number of PCs available, but only if it's relatively painless to get from one OS to the other.
  - fpgaminer 11y ago
    So, there are several ways to handle mouse+keyboard that I've seen. 1) Virtualized Mouse+Keyboard. If you've ever used a VM before, this is what you most likely are familiar with. You click within the VM's window, it captures your mouse and keyboard, and then everything is like normal within the VM. To release, you hit some special key combo and your mouse+keyboard is released back to the host. The VGA Passthrough setup that we're talking about doesn't normally support this method in QEMU/KVM, but you can hack it in. For me, it works as expected, except the mouse+keyboard randomly stop working for brief periods of time every so often. So... not great. 2) Synergy. I've seen this method mentioned a lot, so I guess it works. I haven't tried it. I've heard there are occasional issues, and sometimes Synergy will stop working so you need some kind of fallback method to get back into the Windows VM to get it working again. Probably this method plus method 1 above as a backup will work well. 3) USB passthrough. You could get a second mouse+keyboard, and then use USB filtering to pass them through to the VM. I haven't tried it; it probably works fine if you're okay with having two mouse+keyboard sets on your desk. 4) IOMMU USB passthrough. Similar to #3, but you use IOMMU to give an entire USB hub to the VM. This is likely to have the least number of issues, as it gives direct and exclusive control of a USB hub to the VM. Whether you can do it or not depends on the USB hub configuration on your mobo or whether you feel like adding a USB PCI/PCI-E card. 5) KVM + USB passthrough. Use a KVM and then USB passthrough of some kind (#3 or #4 above). 6) Steam In-Home Streaming. Steam In-Home Streaming can be hacked to share the entire desktop of the target machine. So you could use this to not only handle keyboard+mouse, but also video if you don't want a second monitor and/or don't want to use your existing monitor to switch HDMI/DisplayPort. Since the streaming will be done locally, latency will be very low, but this will burn some CPU/GPU resources. VNC is also an option, but will likely fallover for video games/watching videos/etc.