6 ms·

GVM: A GPU Virtual Machine for IOMMU-Capable Computers

102 points by ArcVRArthur 4y ago

gigatexal 4y ago
Can I use this to create a kvm vm with part of my GPU to run 3D accelerated things?
- ArcVRArthur 4y ago
  Ya, you can definitely do that. Here's our current install documentation: https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi... This will be updated against our GVM/mdev-gpu sources shortly but our current documentation still will get you to what you're asking about. :)
  - gusfrehse 4y ago
    Do you still need to get the VBIOS from the gpu and patch it manually? tbh that was the scariest part
    - ArcVRArthur 4y ago
      On AMD, yes (sadly). On everything else, no.
  - gigatexal 4y ago
    Holy smokes this looks involved. I think I will stick with the wine+proton hoops. I am even planning on selling my nvidia card to go with a RNDA2 GPU so as to have gpu drivers in the kernel. Not taking away from all the work that goes into this -- and I love hacking away as much as the next nerd -- this is just a bit janky, or rather, I'm hesitant to try this on my main workstation.
    - ArcVRArthur 4y ago
      We've got it down to running a script twice and rebooting once then installing your VM. I'm hoping we can simplify a bit further with time. :)
      - gigatexal 4y ago
        Oh you do? Nice! Is the script linked? One concern I had was having to uninstall the Nvidia drivers and used the patched ones?. How does that play with pkg manager updates and things?
        ArcVRArthur 4y ago
        Ya! It's in the install guide. Most package updates don't disrupt anything. Depending on the vendor driver you use (Intel i915, AMD GPU-IOV Module, Nvidia) kernel updates should be okay if we use DKMS. GPU-IOV Module works okay with DKMS if you know what you're doing. I can't speak too much to i915. On Nvidia DKMS is hit or miss but that depends on which driver version is used. Some folks report DKMS doesn't work at all, others say they've never had a problem with it. I think it also depends on the kernel version DKMS is compiling against as in some cases the driver hasn't been updated yet to support the latest kernel version. We try to help with that by making patch files that run ahead of official driver/kernel version support (they basically just make the driver work on a newer kernel - that's all): https://github.com/Arc-Compute/LibVF.IO/tree/master/patches/ https://github.com/Arc-Compute/LibVF.IO/tree/master/patches/
        deleted 4y ago
        [deleted]
        gigatexal 4y ago
        That’s fantastic. Great work by the team. Maybe I’ll find another partition to try this out on.
metadat 4y ago
Can this run a qemu Windows x86 VM?
- ArcVRArthur 4y ago
  Yep! https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi...
ConstantVigil 4y ago
Might I kindly ask where the plan for AMD support comes in, if at all?
- ArcVRArthur 4y ago
  Sadly AMD GPUs currently have some issues. This post has some details about that: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426
shmerl 4y ago
What limits it to Nvidia? And is it using SR-IOV?
- kaladin-jasnah 4y ago
  NVIDIA's vGPU technology has been cracked to work on most Maxwell and later devices. AMD and Intel have not been cracked and researched in the same ways.
  - Namidairo 4y ago
    Intel had GVT-g all the way up to 10th Gen Core, and then claim to have SR-IOV on some 11th Gen processors.
  - shmerl 4y ago
    Yeah, that's what I suspected. I wish AMD would simply support SR-IOV normally.
- ArcVRArthur 4y ago
  We do support other GPUs but currently that's through our LibVF.IO pathway (currently supports Nvidia, Intel, and some AMD GPUs): https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi... A full list of supported devices is available here: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support There are some limitations involved in LibVF.IO such as pre-defined mdev types. GVM is entirely free/libre open source software and it supports arbitrary mdev types. :) We'll do our best to add more supported vendors to GVM/mdev-gpu in the near future.
  - CodesInChaos 4y ago
    Nvidia consumer cards like the 1060 can now be shared between host and VM? I though that was limited to datacenter cards and consumer cards must use exclusive PCI-E passthrough?
    - 4y ago
ncmncm 4y ago
This seems to be at the wrong level of abstraction. I want a virtual Vulkan, one per CPU VM. I think I read somebody was working on that, or something like. That way it works on any GPU, not just NVidia or NVidia/Intel.
- evol262 4y ago
  Looking Glass is kind of the closest you'll get for a trivial "I want shared GPU virtualization on my workstation", but GPU partitioning doesn't really work that way. Outside of the baseline support in the hardware itself, which is nowhere near generic enough for "works on any GPU" (it took supervisory frameworks to even get CUDA/OpenCL/etc to a point where you can stop worrying about writing transforms from scratch and just let PyTorch abstract it a little), this model of GPU partitioning doesn't perform well. How do you allocate vGPU memory between a ML/AI VM, a VDI VM, and a gaming/CAD VM? All have dramatically different requirements. You also can't think of shader/GPU cores in any way similar to CPU cores. They're essentially just vector/linear algebra accelerators with little to no branch prediction, speculative execution, or anything else you'd expect. Otherwise, you can sort of follow along here: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support There's an effort, but it's far from where you want, and there's no indication it will get there unless you can get all the vendors to agree on a standard at some point in the future.
  - ArcVRArthur 4y ago
    I wrote this page over here if you'd like to read more about how vGPUs work (VFIO-Mediated Device/SR-IOV/SIOV): https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals We're doing most of our work (almost all) as open source and we're trying to make sure we have good documentation too. :) Our company website is over here if you're interested to take a look: https://arccompute.io/ https://arccompute.io/
evol262 4y ago
This summary page is absolutely terrible. It appears to be an open-source implementation of nvidia-cli for managing mdevs, which is arbitrarily nice, but it's not clear to end-users what this means. Pre-Ampere GPUs (Ampere+ is MIG) were able to use the mediated device subsystem to partition cards into time slices. It's similar to SR-IOV, except that you can specify the size of the partition with more granularity than "give me a new virtual device". Intel has GVT-g, which is reasonably widely supported on Gen12 and above (edit: too late/early -- Gen11 and earlier, not Gen12 and later -- thanks to my123 for the correction). nVidia had vGPU/mdev, and newer generations (Ampere and later) use MIG. It's unclear whether this supports MIG at all. AMD uses MxGPU, and they've never really cared about/pursued anything related to this, probably because their datacenter penetration is about 1%. MxGPU is only supported on some FirePro cards. mdev was largely on GRID cards (mostly Tesla, some Quadros). MIG is on AXXX cards. It's unclear why anyone should use this over mdevctl, which already supports GVT-g, and it's also unclear whether this is tied to the (very much "don't use in production") open source nvidia drivers. For end-users, GVT-g, getting a cheap older GRID card, or using Looking Glass for your GPU are all more reasonable options. This effort is great, but the readme is appallingly short on information even for someone who knows the problem domain.
- kaladin-jasnah 4y ago
  Another option is getting any Kepler (GK107/GK104) series card (~$10) and running Xen and an unlocker. GVT-g or a new GPU is probably a better solution, though. Are you sure GVT-g is Gen12 though? I always thought Intel switched to SRIOV by then.
  - my123 4y ago
    > GVT-g or a new GPU is probably a better solution, though. Are you sure GVT-g is Gen12 though? I always thought Intel switched to SRIOV by then. Currently, for Gen11 and later (in Ice Lake), neither GVT-g nor SR-IOV-based GPU virt options are available. As we're several years past Gen11 launch, seems that Intel isn't interested in GPU virtualisation much anymore on a Linux host.
    - 4y ago
rob_c 4y ago
no AMD, why for Intel and doesn't cover 90% of nvidia products most people will have (unles they own and run a datacenter)... which leads me to ask... why even bother?
- ArcVRArthur 4y ago
  This works on most Nvidia consumer cards (not just datacenter cards): https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support I think we have decent coverage of device support: https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw... We'd like to improve support for AMD devices but there are some issues there that have yet to be resolved.
  - rob_c 4y ago
    Seriously, no. And this isn't just an unhelpful comment on a forum. This is serious advice from an expert here. After patching the Nvidia driver and then patching the driver itself to allow other non-standard vgpu definitions what's the point of this? Its at best a feel good that I reimplemented a tiny part. You'd be better contributing patches against binary blobs to the community rather than wasting time on this.
    - ArcVRArthur 4y ago
      Nearest I can tell the guy earlier thought what mdevctl is (echo uuid >> /sys/mdev_bus/$pci-device/available-types/) is the same as what we are doing. That couldn't be further from the truth. I tried explaining to him we arbitrarily define the available types on behalf of the vendor driver via user configuration. It’s the difference between picking from a pre-made list of things someone else says you’re allowed to do and making the list of things you’re allowed to do yourself.

102 points by ArcVRArthur 4y ago

gigatexal 4y ago

Can I use this to create a kvm vm with part of my GPU to run 3D accelerated things?

ArcVRArthur 4y ago
Ya, you can definitely do that. Here's our current install documentation: https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi... This will be updated against our GVM/mdev-gpu sources shortly but our current documentation still will get you to what you're asking about. :)
- gusfrehse 4y ago
  Do you still need to get the VBIOS from the gpu and patch it manually? tbh that was the scariest part
  - ArcVRArthur 4y ago
    On AMD, yes (sadly). On everything else, no.
- gigatexal 4y ago
  Holy smokes this looks involved. I think I will stick with the wine+proton hoops. I am even planning on selling my nvidia card to go with a RNDA2 GPU so as to have gpu drivers in the kernel. Not taking away from all the work that goes into this -- and I love hacking away as much as the next nerd -- this is just a bit janky, or rather, I'm hesitant to try this on my main workstation.
  - ArcVRArthur 4y ago
    We've got it down to running a script twice and rebooting once then installing your VM. I'm hoping we can simplify a bit further with time. :)
    - gigatexal 4y ago
      Oh you do? Nice! Is the script linked? One concern I had was having to uninstall the Nvidia drivers and used the patched ones?. How does that play with pkg manager updates and things?
      - ArcVRArthur 4y ago
        Ya! It's in the install guide. Most package updates don't disrupt anything. Depending on the vendor driver you use (Intel i915, AMD GPU-IOV Module, Nvidia) kernel updates should be okay if we use DKMS. GPU-IOV Module works okay with DKMS if you know what you're doing. I can't speak too much to i915. On Nvidia DKMS is hit or miss but that depends on which driver version is used. Some folks report DKMS doesn't work at all, others say they've never had a problem with it. I think it also depends on the kernel version DKMS is compiling against as in some cases the driver hasn't been updated yet to support the latest kernel version. We try to help with that by making patch files that run ahead of official driver/kernel version support (they basically just make the driver work on a newer kernel - that's all): https://github.com/Arc-Compute/LibVF.IO/tree/master/patches/ https://github.com/Arc-Compute/LibVF.IO/tree/master/patches/
        deleted 4y ago
        [deleted]
        gigatexal 4y ago
        That’s fantastic. Great work by the team. Maybe I’ll find another partition to try this out on.

metadat 4y ago

Can this run a qemu Windows x86 VM?

ArcVRArthur 4y ago
Yep! https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi...

ConstantVigil 4y ago

Might I kindly ask where the plan for AMD support comes in, if at all?

ArcVRArthur 4y ago
Sadly AMD GPUs currently have some issues. This post has some details about that: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426

shmerl 4y ago

What limits it to Nvidia? And is it using SR-IOV?

kaladin-jasnah 4y ago
NVIDIA's vGPU technology has been cracked to work on most Maxwell and later devices. AMD and Intel have not been cracked and researched in the same ways.
- Namidairo 4y ago
  Intel had GVT-g all the way up to 10th Gen Core, and then claim to have SR-IOV on some 11th Gen processors.
- shmerl 4y ago
  Yeah, that's what I suspected. I wish AMD would simply support SR-IOV normally.
ArcVRArthur 4y ago
We do support other GPUs but currently that's through our LibVF.IO pathway (currently supports Nvidia, Intel, and some AMD GPUs): https://arccompute.com/blog/libvfio-commodity-gpu-multiplexing/ https://arccompute.com/blog/libvfio-commodity-gpu-multiplexi... A full list of supported devices is available here: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support There are some limitations involved in LibVF.IO such as pre-defined mdev types. GVM is entirely free/libre open source software and it supports arbitrary mdev types. :) We'll do our best to add more supported vendors to GVM/mdev-gpu in the near future.
- CodesInChaos 4y ago
  Nvidia consumer cards like the 1060 can now be shared between host and VM? I though that was limited to datacenter cards and consumer cards must use exclusive PCI-E passthrough?
  - 4y ago

ncmncm 4y ago

This seems to be at the wrong level of abstraction. I want a virtual Vulkan, one per CPU VM. I think I read somebody was working on that, or something like. That way it works on any GPU, not just NVidia or NVidia/Intel.

evol262 4y ago
Looking Glass is kind of the closest you'll get for a trivial "I want shared GPU virtualization on my workstation", but GPU partitioning doesn't really work that way. Outside of the baseline support in the hardware itself, which is nowhere near generic enough for "works on any GPU" (it took supervisory frameworks to even get CUDA/OpenCL/etc to a point where you can stop worrying about writing transforms from scratch and just let PyTorch abstract it a little), this model of GPU partitioning doesn't perform well. How do you allocate vGPU memory between a ML/AI VM, a VDI VM, and a gaming/CAD VM? All have dramatically different requirements. You also can't think of shader/GPU cores in any way similar to CPU cores. They're essentially just vector/linear algebra accelerators with little to no branch prediction, speculative execution, or anything else you'd expect. Otherwise, you can sort of follow along here: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support There's an effort, but it's far from where you want, and there's no indication it will get there unless you can get all the vendors to agree on a standard at some point in the future.
- ArcVRArthur 4y ago
  I wrote this page over here if you'd like to read more about how vGPUs work (VFIO-Mediated Device/SR-IOV/SIOV): https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals We're doing most of our work (almost all) as open source and we're trying to make sure we have good documentation too. :) Our company website is over here if you're interested to take a look: https://arccompute.io/ https://arccompute.io/

evol262 4y ago

This summary page is absolutely terrible. It appears to be an open-source implementation of nvidia-cli for managing mdevs, which is arbitrarily nice, but it's not clear to end-users what this means. Pre-Ampere GPUs (Ampere+ is MIG) were able to use the mediated device subsystem to partition cards into time slices. It's similar to SR-IOV, except that you can specify the size of the partition with more granularity than "give me a new virtual device". Intel has GVT-g, which is reasonably widely supported on Gen12 and above (edit: too late/early -- Gen11 and earlier, not Gen12 and later -- thanks to my123 for the correction). nVidia had vGPU/mdev, and newer generations (Ampere and later) use MIG. It's unclear whether this supports MIG at all. AMD uses MxGPU, and they've never really cared about/pursued anything related to this, probably because their datacenter penetration is about 1%. MxGPU is only supported on some FirePro cards. mdev was largely on GRID cards (mostly Tesla, some Quadros). MIG is on AXXX cards. It's unclear why anyone should use this over mdevctl, which already supports GVT-g, and it's also unclear whether this is tied to the (very much "don't use in production") open source nvidia drivers. For end-users, GVT-g, getting a cheap older GRID card, or using Looking Glass for your GPU are all more reasonable options. This effort is great, but the readme is appallingly short on information even for someone who knows the problem domain.

kaladin-jasnah 4y ago
Another option is getting any Kepler (GK107/GK104) series card (~$10) and running Xen and an unlocker. GVT-g or a new GPU is probably a better solution, though. Are you sure GVT-g is Gen12 though? I always thought Intel switched to SRIOV by then.
- my123 4y ago
  > GVT-g or a new GPU is probably a better solution, though. Are you sure GVT-g is Gen12 though? I always thought Intel switched to SRIOV by then. Currently, for Gen11 and later (in Ice Lake), neither GVT-g nor SR-IOV-based GPU virt options are available. As we're several years past Gen11 launch, seems that Intel isn't interested in GPU virtualisation much anymore on a Linux host.
  - 4y ago

rob_c 4y ago

no AMD, why for Intel and doesn't cover 90% of nvidia products most people will have (unles they own and run a datacenter)... which leads me to ask... why even bother?

ArcVRArthur 4y ago
This works on most Nvidia consumer cards (not just datacenter cards): https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support I think we have decent coverage of device support: https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam https://store.steampowered.com/hwsurvey/Steam-Hardware-Softw... We'd like to improve support for AMD devices but there are some issues there that have yet to be resolved.
- rob_c 4y ago
  Seriously, no. And this isn't just an unhelpful comment on a forum. This is serious advice from an expert here. After patching the Nvidia driver and then patching the driver itself to allow other non-standard vgpu definitions what's the point of this? Its at best a feel good that I reimplemented a tiny part. You'd be better contributing patches against binary blobs to the community rather than wasting time on this.
  - ArcVRArthur 4y ago
    Nearest I can tell the guy earlier thought what mdevctl is (echo uuid >> /sys/mdev_bus/$pci-device/available-types/) is the same as what we are doing. That couldn't be further from the truth. I tried explaining to him we arbitrarily define the available types on behalf of the vendor driver via user configuration. It’s the difference between picking from a pre-made list of things someone else says you’re allowed to do and making the list of things you’re allowed to do yourself.