5 ms·

How much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? Is it as simple as it seems? I'd love to switch over

by joshAg 11y ago

How much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? Is it as simple as it seems? I'd love to switch over to a VM for windows games backed by vtx and vtd, but I've been worried that it'd be waay more work.

swinglock 11y ago
Keep in mind there's more to it than pure frames per second. You also got latency and jitter to worry about which is vital but almost always ignored, probably because it's difficult to measure and understand. Doing it properly still required special hardware and software I believe.
- fpgaminer 11y ago
  Indeed, I've seen some talk about latency/jitter issues on the vfio mailing list. I haven't tested it myself yet, so we'll see how it goes. Though I figure things will just naturally get better over time as the kernel's support improves.
miffe 11y ago
I tried this a while back. My results can be seen at http://miffe.org/bench/ http://miffe.org/bench/ The largest problem is that the NVIDIA driver refuses to work when it detects that it's in a VM. The workaround is to hide the VM status for the guest os, by using -cpu host,kvm=off and disabling the tweaks that makes windows work better (hv_time,hv_relaxed,hv_vapic).
- Diederich 11y ago
  If you made a nice blog post about this, in detail, and shared it, a lot of folks would find it extremely interesting!
- fpgaminer 11y ago
  Wow, that's a great resource! I'm glad to see your disk benchmarks were also all over the place. I couldn't seem to make much sense of CrystalMark's results, even when increasing the test size above my RAM size to eliminate caching. I'm using virtio to an SSD dedicated to the VM and saw, like you, some markers were better in the VM than they should be, and some where significantly worse. Very weird. Luckily I won't matter for gaming.
- driverdan 11y ago
  A better workaround would be to buy AMD GPUs.
  - miffe 11y ago
    Thats my plan. I got the GTX 980 for free, so I'm using until the next gen AMD cards are released.
  - redtuesday 11y ago
    I always wonder why they don't really advertise this to gamers as a feature at which they are better than nvidia.
  - deleted 11y ago
    [deleted]
fpgaminer 11y ago
> How's much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? I've seen research that shows the GPU being 98-99% when using QEMU/KVM. I haven't used the system long enough to accurately quantify my personal experience. > Is it as simple as it seems? For me getting the GPU working was the easiest part. It worked on the first try. The hardest part was finding good docs (Arch wiki & VFIO blog), trying to understand their little ecosystem, and finding the right compatible parts (hint: besides the CPU, it's really easy). I'd never used QEMU/KVM before (I used VirtualBox previously) so that was a big learning curve. It seems like things have matured quite a bit since the VFIO blog wrote its articles, as a number of things had changed and become easier.
- hbosch 11y ago
  Does "getting the GPU to work" mean you can give your Windows VM exclusive rights to the ports on the back of that graphics card as well? That would be amazing. I'd love to, say, run a Hackintosh setup with Windows 10 in a VM that could fully utilize it's own GPU + external ports -- this way I could use a modern VR setup (Oculus or Vive) without having to use Windows full time!
  - fpgaminer 11y ago
    In fact, that's the only way to do it. This VFIO/IOMMU setup virtualizes the PCI-E port itself. It routes the PCI-E port that the graphics card is plugged into to the VM. So the VM has exclusive and direct access to that PCI-E port. Therefore, the GPU, the ports on it, everything hooked up to that PCI-E port belongs to the VM. Most importantly, this also means the GPU's SMBus for temperature monitoring is also routed and owned by the VM. From the VM's perspective it isn't any different than a physical card. You can also route USB hubs to it the same way, sound cards, etc.
    - hbosch 11y ago
      I can't believe I've never heard of this before. I'm honestly astonished! Thanks for your reply, I can't wait to try this.

by joshAg 11y ago

swinglock 11y ago

Keep in mind there's more to it than pure frames per second. You also got latency and jitter to worry about which is vital but almost always ignored, probably because it's difficult to measure and understand. Doing it properly still required special hardware and software I believe.

fpgaminer 11y ago
Indeed, I've seen some talk about latency/jitter issues on the vfio mailing list. I haven't tested it myself yet, so we'll see how it goes. Though I figure things will just naturally get better over time as the kernel's support improves.

miffe 11y ago

I tried this a while back. My results can be seen at http://miffe.org/bench/ http://miffe.org/bench/ The largest problem is that the NVIDIA driver refuses to work when it detects that it's in a VM. The workaround is to hide the VM status for the guest os, by using -cpu host,kvm=off and disabling the tweaks that makes windows work better (hv_time,hv_relaxed,hv_vapic).

Diederich 11y ago
If you made a nice blog post about this, in detail, and shared it, a lot of folks would find it extremely interesting!
fpgaminer 11y ago
Wow, that's a great resource! I'm glad to see your disk benchmarks were also all over the place. I couldn't seem to make much sense of CrystalMark's results, even when increasing the test size above my RAM size to eliminate caching. I'm using virtio to an SSD dedicated to the VM and saw, like you, some markers were better in the VM than they should be, and some where significantly worse. Very weird. Luckily I won't matter for gaming.
driverdan 11y ago
A better workaround would be to buy AMD GPUs.
- miffe 11y ago
  Thats my plan. I got the GTX 980 for free, so I'm using until the next gen AMD cards are released.
- redtuesday 11y ago
  I always wonder why they don't really advertise this to gamers as a feature at which they are better than nvidia.
- deleted 11y ago
  [deleted]

fpgaminer 11y ago

> How's much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? I've seen research that shows the GPU being 98-99% when using QEMU/KVM. I haven't used the system long enough to accurately quantify my personal experience. > Is it as simple as it seems? For me getting the GPU working was the easiest part. It worked on the first try. The hardest part was finding good docs (Arch wiki & VFIO blog), trying to understand their little ecosystem, and finding the right compatible parts (hint: besides the CPU, it's really easy). I'd never used QEMU/KVM before (I used VirtualBox previously) so that was a big learning curve. It seems like things have matured quite a bit since the VFIO blog wrote its articles, as a number of things had changed and become easier.

hbosch 11y ago
Does "getting the GPU to work" mean you can give your Windows VM exclusive rights to the ports on the back of that graphics card as well? That would be amazing. I'd love to, say, run a Hackintosh setup with Windows 10 in a VM that could fully utilize it's own GPU + external ports -- this way I could use a modern VR setup (Oculus or Vive) without having to use Windows full time!
- fpgaminer 11y ago
  In fact, that's the only way to do it. This VFIO/IOMMU setup virtualizes the PCI-E port itself. It routes the PCI-E port that the graphics card is plugged into to the VM. So the VM has exclusive and direct access to that PCI-E port. Therefore, the GPU, the ports on it, everything hooked up to that PCI-E port belongs to the VM. Most importantly, this also means the GPU's SMBus for temperature monitoring is also routed and owned by the VM. From the VM's perspective it isn't any different than a physical card. You can also route USB hubs to it the same way, sound cards, etc.
  - hbosch 11y ago
    I can't believe I've never heard of this before. I'm honestly astonished! Thanks for your reply, I can't wait to try this.