10 ms·

LibVF.IO: Add support for GPU Virtual Machine (GVM)

118 points by ArcVRArthur 4y ago

ArcVRArthur 4y ago
Hey, I'm the co-creator of LibVF.IO and the GVM project. I'd love to hear what you think about our latest release where we added support for GVM components in LibVF.IO. Side note: We also added in support for Libvirt by allowing users to install standalone GVM components via ./scripts/install-standalone-gvm-components.sh.
- gorkish 4y ago
  To be honest with you I am extremely excited by what you are working towards, but utterly beyond confusion when it comes to the GPU virtualization landscape. The walls that vendors have built in this area in their attempt to artificially segment "consumer" and "professional" are shameful. NVidia in particular has been reprehensible. I wish more people cared. Thank you!
  - ArcVRArthur 4y ago
    Thanks for your kind words! We're all virtualization users ourselves so I definitely get what you mean - we started this project because we wanted to help people to get access to the magic at home using the hardware they already have! I will say though that Nvidia has done a lot towards improving their support for open source. While it might be a bit of an uncommon take in my view many of the folks who probably are most passionate about bringing exciting GPU technology to the world work at Nvidia. :)
    - DennisAleynikov 4y ago
      Gigachad Arthur
- MuffinFlavored 4y ago
  https://github.com/Arc-Compute/LibVF.IO https://github.com/Arc-Compute/LibVF.IO https://docs.linux-gvm.org/ https://docs.linux-gvm.org/ For anybody else interested
- deleted 4y ago
  [deleted]
- elcritch 4y ago
  Awesome project! And I gotta say thats some pretty code for dealing with a bunch of sysfs stuff: https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libvfio/control/iommu.nim https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libv...
  - ArcVRArthur 4y ago
    Thanks!! We do our best to keep the code as clean/readable as possible. The first version was a bit of a mess but we rewrote it again clean slate to improve over the first implementation. :)
lostmsu 4y ago
What does it do? Does it let one pass a virtual GPU to the guest without complete PCI-passthrough? E.g. will I be able to share host GPU between the host system and guests?
- ArcVRArthur 4y ago
  Ya, that's exactly correct. :)
  - Melatonic 4y ago
    What about one GPU with multiple VM? Annoying t hing right now is that you have to pony up for the fancy nvidia cards plus insane nvidia licensing for their multiGPU virtualized cards. AMD at least makes you only pay for the hardware but its still super pricey.
    - ArcVRArthur 4y ago
      Ya, you can use multiple VMs. Here is a video I took of a laptop with LibVF.IO and an early pre-release version of GVM installed - at the start of the video you can see a fullscreen Windows VM and then I back out and open another compartmentalized 'app VM' (browser): https://streamable.com/gd45bh https://streamable.com/gd45bh Both VMs are actually based on Windows (one simply doesn't have explorer.exe running) and each has it's own virtual GPU attached. This laptop has a Nvidia graphics processor inside but it also works with laptops that have an Xe Intel graphics processor. I'm hoping at some point it will work well on AMD too.
      - lostmsu 4y ago
        Can I run CUDA in the guest OS (Windows)?
      - kbumsik 4y ago
        Sounds very interesting! Do VMs work with a display session (Xorg) too? Some GPU-enabled applications require an X server with a physical HDMI/Displayport attached.
synergy20 4y ago
GPU can not even run its own OS basically, it's a parallel co-processor, when-should/how-could I use it as a VM?
- ArcVRArthur 4y ago
  According to https://docs.linux-gvm.org/ https://docs.linux-gvm.org/ "GVM ... may be used in combination with KVM or other platform hypervisors such as Xen* to provide a complete virtualization solution via both central processing (CPU) and graphics processing (GPU) hardware acceleration."
- fulafel 4y ago
  Most current GPUs could, even with the public programming info. The limitations in interrupt handling would eat into either functionality or performance somewhat I think.
maldev 4y ago
This is pretty cool, one of the killer features that windows has had with Hyper-V that is really creaming QEMU/KVM. You could probably try getting some support from a vendor like Red Hat, as something like this in the Linux Kernel is a huge datacenter/cloud hosting feature that directly impacts their bottom line.
- ArcVRArthur 4y ago
  I would love to work with folks like the people at RedHat! We're working on some new tools in C (GPLv2) which we think will make it easier to work with existing kernel/driver maintainers. We'll announce some of those things early next month and right after we'll be at QubesOS Summit & KVM Forum. I hope we get the opportunity to talk to some of the RedHat folks! :)
  - csdreamer7 4y ago
    Do you have a contact where Red Hat people can reach you? (Not in the VM/Graphics space, but I can try to reach out to the right people.)
    - ArcVRArthur 4y ago
      For sure! They can reach me at arthur@arccompute.io I'll also be attending KVM Forum this year so I'd love to chat with folks there as well! :)
habobobo 4y ago
Is it possible to use this for a remote gpu solution somehow?
- ArcVRArthur 4y ago
  That depends on the driver implementation and what exactly the features are that you are looking for. In general if you're using virtualization tools like KVM/Xen you can configure a computer to make virtual machines that can be accessed remotely (for example you would likely want to configure the network to provide your machine with an IP address). You can use GVM for GPU accelerating various programs like those that use DirectX, Vulkan, and OpenGL so if that meets your needs from a remote machine and you have the necessary resources (networking resources such as IP addresses, time and know-how to configure such a setup) then I would say yes.
yissp 4y ago
Is this similar to the virglrenderer project? https://gitlab.freedesktop.org/virgl/virglrenderer https://gitlab.freedesktop.org/virgl/virglrenderer
- ArcVRArthur 4y ago
  It is slightly different - this runs an unmodified GPU driver in the guest (no para-virtualization) and in our observations our performance is close to native which I believe is a fair improvement over the overhead of Virgl. Also there's availability of more APIs - for example DirectX, Vulkan, and OpenGL work with GVM guests. I don't think all of those work with Virgl. We make use of various different assistance modes (either hardware or software I/O virtualization/resource sharing) via VFIO-Mdev (VFIO Mediated Device), or via SR-IOV (Single Root IO Virtualization), or SIOV (Scalable IO Virtualization). I made this web page to try consolidate some information from various folks who have contributed a lot in this area of open source to show how it works (there are some novel additions we've made as well based on our own work with GVM): https://openmdev.io/index.php/Virtual_IO_Internals https://openmdev.io/index.php/Virtual_IO_Internals
punkpeye 4y ago
What are the practical use cases for this?
- SXX 4y ago
  Practical use would be rather niche. For instance easy splitting resources of single GPU between multiple VMs if you want to run isolated instances of ML software. Or if you're need to run automated tests for your game across different OS. Or tests for any software that use GPU acceleration. In reality most users of VFIO passthrough use it for gaming or creating multiseat systems. I guess it's will be true for vGPU feature too.
  - ArcVRArthur 4y ago
    There are also some benefits in load balancing GPUs. For instance processes can be controlled with controlled groups (cgroups) and niceness/thread affinity to make sure scheduling / resource allocation is done equitably between processes on the system. Using GVM and 'app VMs' it is possible to do similar things with the GPU (restrict GPU processes to a user configurable slice of the GPU so it cannot deny service to other processes on the system which are also using the GPU): https://openmdev.io/index.php/Mdev-GPU#fbLen https://openmdev.io/index.php/Mdev-GPU#fbLen Since this type of load balancing was originally used in the datacenter where virtual machine multi-tenancy was the use case the Quality of Service (QoS) functions here are fairly robust.
- ArcVRArthur 4y ago
  For my use I tend to run a high performing Windows VM (very good with most games and other GPU accelerated programs - even chrome/adobe these days uses the GPU) inside Linux. This can also be used for various high assurance use cases such as OpenXT / QubesOS (security by compartmentalization). For example a laptop computer being used by an automotive company for CAD to keep their information safe from other programs on the system (like a browser in another VM) the prying eyes of competitors. I made a video of that here: https://news.ycombinator.com/item?id=32585333 https://news.ycombinator.com/item?id=32585333 We have also been working on something called "LIME Is Mediated Emulation" which is a Win64 binary compatibility layer similar to WINE but using vGPUs. You can read about that here: https://openmdev.io/index.php/LIME_Is_Mediated_Emulation https://openmdev.io/index.php/LIME_Is_Mediated_Emulation
stuaxo 4y ago
Can I use this to with an AMD APU, like the graphics on a Ryzen ?
- ArcVRArthur 4y ago
  Unfortunately there is limited support on AMD. I would like to try to make it better though! https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support
  - MuffinFlavored 4y ago
    What's one of the biggest things AMD could go to help you turn this into a reality? Just... documentation? Source code examples? Which... I'm assuming they'll never do?
    - ArcVRArthur 4y ago
      If they would consider helping with the things I mention here we'd do our best to support them! https://news.ycombinator.com/item?id=32585950 https://news.ycombinator.com/item?id=32585950
rcarmo 4y ago
I really want this to work on AMD hardware.
- ArcVRArthur 4y ago
  I hope I can work with them some time to improve support! I outlined a few things they would need to do to help support GVM using the amdgpu driver on this page: https://openmdev.io/index.php/AMDGPU https://openmdev.io/index.php/AMDGPU Ideally some folks who know about amdgpu might consider helping our open source community by adding similar information to that page to the information we added on the Nvidia Open Kernel Modules page: https://openmdev.io/index.php/OpenRM https://openmdev.io/index.php/OpenRM If that could be done then we would do our best to add in AMD support to GVM.
montecarl 4y ago
I run Linux for software development, but I need to also run Autodesk Fusion 360, which doesn't run on Linux. So I run it in a VirtualBox virtual machine. It works poorly as it sometimes doesn't render correctly and the performance is bad. I know that if I did some research and bought a second GPU that I could maybe get PCI-passthrough working, but I find that obnoxious and wasteful. Can I use LibVF.IO/GVM to get better performance (and perhaps proper rendering) on a VM in Linux? Or is there a different solution I should be looking into?
- ArcVRArthur 4y ago
  Ya, LibVF.IO & GVM are built for things like this! For example I have a friend who uses it for various Adobe programs which also don't work well on Linux.
- amarshall 4y ago
  It depends if your GPU is supported, and such support is mostly beholden to the hardware before anything else. Currently it’s largely limited to: for Nvidia, most Maxwell, Pascal, & Turing cards; for AMD, very few obsolete cards; for Intel, gen 6–9 iGPUs.
  - ArcVRArthur 4y ago
    We have Ampere support working on some devices (and it is in development on other Ampere devices) as well as 11th & 12th generation Intel Xe. :) Here's the GPU Support page if you'd like to take a look: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support
    - Scramblejams 4y ago
      Love this project. I nerded out on GPU passthrough several years ago and learned a lot. GPU sharing would be a killer feature for some of my workflows. Recently, after being a long-time Nvidia customer, for Linux systems I've switched to AMD GPUs and have been very impressed with the amdgpu driver quality relative to Nvidia's proprietary driver. Do you have any thoughts on what the LibVF/GVM future looks like for AMD devices?
deleted 4y ago
[deleted]
Teever 4y ago
Can I use this with Proxmox? I looked into LibVF.IO when it was last posted on hackernews but I was never able to get it to work with Proxmox. I'm kind of a weirdo and want to use proxmox as a hypervisor on my desktop and manjaro as my daily driver with Windows 10 for Fusion360.
ajvs 4y ago
What's the recommended GPUs with best performance:cost?
- ArcVRArthur 4y ago
  That depends on your use-case. In general I would recommend you consider purchasing Nvidia's GPUs for the best price/performance and GVM support. Intel's Xe architecture is currently improving but the performance isn't quite there for a number of use-cases however some appear to work quite well and I expect that will improve with time. The 2080Ti works well with current software. If you are a developer and would like to help us improve support for devices you can purchase a 3090Ti (support in GVM for this device is under active development).
deleted 4y ago
[deleted]

118 points by ArcVRArthur 4y ago

Hey, I'm the co-creator of LibVF.IO and the GVM project. I'd love to hear what you think about our latest release where we added support for GVM components in LibVF.IO. Side note: We also added in support for Libvirt by allowing users to install standalone GVM components via ./scripts/install-standalone-gvm-components.sh.

gorkish 4y ago
To be honest with you I am extremely excited by what you are working towards, but utterly beyond confusion when it comes to the GPU virtualization landscape. The walls that vendors have built in this area in their attempt to artificially segment "consumer" and "professional" are shameful. NVidia in particular has been reprehensible. I wish more people cared. Thank you!
- ArcVRArthur 4y ago
  Thanks for your kind words! We're all virtualization users ourselves so I definitely get what you mean - we started this project because we wanted to help people to get access to the magic at home using the hardware they already have! I will say though that Nvidia has done a lot towards improving their support for open source. While it might be a bit of an uncommon take in my view many of the folks who probably are most passionate about bringing exciting GPU technology to the world work at Nvidia. :)
  - DennisAleynikov 4y ago
    Gigachad Arthur
MuffinFlavored 4y ago
https://github.com/Arc-Compute/LibVF.IO https://github.com/Arc-Compute/LibVF.IO https://docs.linux-gvm.org/ https://docs.linux-gvm.org/ For anybody else interested
deleted 4y ago
[deleted]
elcritch 4y ago
Awesome project! And I gotta say thats some pretty code for dealing with a bunch of sysfs stuff: https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libvfio/control/iommu.nim https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libv...
- ArcVRArthur 4y ago
  Thanks!! We do our best to keep the code as clean/readable as possible. The first version was a bit of a mess but we rewrote it again clean slate to improve over the first implementation. :)

lostmsu 4y ago

What does it do? Does it let one pass a virtual GPU to the guest without complete PCI-passthrough? E.g. will I be able to share host GPU between the host system and guests?

ArcVRArthur 4y ago
Ya, that's exactly correct. :)
- Melatonic 4y ago
  What about one GPU with multiple VM? Annoying t hing right now is that you have to pony up for the fancy nvidia cards plus insane nvidia licensing for their multiGPU virtualized cards. AMD at least makes you only pay for the hardware but its still super pricey.
  - ArcVRArthur 4y ago
    Ya, you can use multiple VMs. Here is a video I took of a laptop with LibVF.IO and an early pre-release version of GVM installed - at the start of the video you can see a fullscreen Windows VM and then I back out and open another compartmentalized 'app VM' (browser): https://streamable.com/gd45bh https://streamable.com/gd45bh Both VMs are actually based on Windows (one simply doesn't have explorer.exe running) and each has it's own virtual GPU attached. This laptop has a Nvidia graphics processor inside but it also works with laptops that have an Xe Intel graphics processor. I'm hoping at some point it will work well on AMD too.
    - lostmsu 4y ago
      Can I run CUDA in the guest OS (Windows)?
    - kbumsik 4y ago
      Sounds very interesting! Do VMs work with a display session (Xorg) too? Some GPU-enabled applications require an X server with a physical HDMI/Displayport attached.

synergy20 4y ago

GPU can not even run its own OS basically, it's a parallel co-processor, when-should/how-could I use it as a VM?

ArcVRArthur 4y ago
According to https://docs.linux-gvm.org/ https://docs.linux-gvm.org/ "GVM ... may be used in combination with KVM or other platform hypervisors such as Xen* to provide a complete virtualization solution via both central processing (CPU) and graphics processing (GPU) hardware acceleration."
fulafel 4y ago
Most current GPUs could, even with the public programming info. The limitations in interrupt handling would eat into either functionality or performance somewhat I think.

maldev 4y ago

This is pretty cool, one of the killer features that windows has had with Hyper-V that is really creaming QEMU/KVM. You could probably try getting some support from a vendor like Red Hat, as something like this in the Linux Kernel is a huge datacenter/cloud hosting feature that directly impacts their bottom line.

ArcVRArthur 4y ago
I would love to work with folks like the people at RedHat! We're working on some new tools in C (GPLv2) which we think will make it easier to work with existing kernel/driver maintainers. We'll announce some of those things early next month and right after we'll be at QubesOS Summit & KVM Forum. I hope we get the opportunity to talk to some of the RedHat folks! :)
- csdreamer7 4y ago
  Do you have a contact where Red Hat people can reach you? (Not in the VM/Graphics space, but I can try to reach out to the right people.)
  - ArcVRArthur 4y ago
    For sure! They can reach me at arthur@arccompute.io I'll also be attending KVM Forum this year so I'd love to chat with folks there as well! :)

habobobo 4y ago

Is it possible to use this for a remote gpu solution somehow?

ArcVRArthur 4y ago
That depends on the driver implementation and what exactly the features are that you are looking for. In general if you're using virtualization tools like KVM/Xen you can configure a computer to make virtual machines that can be accessed remotely (for example you would likely want to configure the network to provide your machine with an IP address). You can use GVM for GPU accelerating various programs like those that use DirectX, Vulkan, and OpenGL so if that meets your needs from a remote machine and you have the necessary resources (networking resources such as IP addresses, time and know-how to configure such a setup) then I would say yes.

yissp 4y ago

Is this similar to the virglrenderer project? https://gitlab.freedesktop.org/virgl/virglrenderer https://gitlab.freedesktop.org/virgl/virglrenderer

ArcVRArthur 4y ago
It is slightly different - this runs an unmodified GPU driver in the guest (no para-virtualization) and in our observations our performance is close to native which I believe is a fair improvement over the overhead of Virgl. Also there's availability of more APIs - for example DirectX, Vulkan, and OpenGL work with GVM guests. I don't think all of those work with Virgl. We make use of various different assistance modes (either hardware or software I/O virtualization/resource sharing) via VFIO-Mdev (VFIO Mediated Device), or via SR-IOV (Single Root IO Virtualization), or SIOV (Scalable IO Virtualization). I made this web page to try consolidate some information from various folks who have contributed a lot in this area of open source to show how it works (there are some novel additions we've made as well based on our own work with GVM): https://openmdev.io/index.php/Virtual_IO_Internals https://openmdev.io/index.php/Virtual_IO_Internals

punkpeye 4y ago

What are the practical use cases for this?

SXX 4y ago
Practical use would be rather niche. For instance easy splitting resources of single GPU between multiple VMs if you want to run isolated instances of ML software. Or if you're need to run automated tests for your game across different OS. Or tests for any software that use GPU acceleration. In reality most users of VFIO passthrough use it for gaming or creating multiseat systems. I guess it's will be true for vGPU feature too.
- ArcVRArthur 4y ago
  There are also some benefits in load balancing GPUs. For instance processes can be controlled with controlled groups (cgroups) and niceness/thread affinity to make sure scheduling / resource allocation is done equitably between processes on the system. Using GVM and 'app VMs' it is possible to do similar things with the GPU (restrict GPU processes to a user configurable slice of the GPU so it cannot deny service to other processes on the system which are also using the GPU): https://openmdev.io/index.php/Mdev-GPU#fbLen https://openmdev.io/index.php/Mdev-GPU#fbLen Since this type of load balancing was originally used in the datacenter where virtual machine multi-tenancy was the use case the Quality of Service (QoS) functions here are fairly robust.
ArcVRArthur 4y ago
For my use I tend to run a high performing Windows VM (very good with most games and other GPU accelerated programs - even chrome/adobe these days uses the GPU) inside Linux. This can also be used for various high assurance use cases such as OpenXT / QubesOS (security by compartmentalization). For example a laptop computer being used by an automotive company for CAD to keep their information safe from other programs on the system (like a browser in another VM) the prying eyes of competitors. I made a video of that here: https://news.ycombinator.com/item?id=32585333 https://news.ycombinator.com/item?id=32585333 We have also been working on something called "LIME Is Mediated Emulation" which is a Win64 binary compatibility layer similar to WINE but using vGPUs. You can read about that here: https://openmdev.io/index.php/LIME_Is_Mediated_Emulation https://openmdev.io/index.php/LIME_Is_Mediated_Emulation

stuaxo 4y ago

Can I use this to with an AMD APU, like the graphics on a Ryzen ?

ArcVRArthur 4y ago
Unfortunately there is limited support on AMD. I would like to try to make it better though! https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support
- MuffinFlavored 4y ago
  What's one of the biggest things AMD could go to help you turn this into a reality? Just... documentation? Source code examples? Which... I'm assuming they'll never do?
  - ArcVRArthur 4y ago
    If they would consider helping with the things I mention here we'd do our best to support them! https://news.ycombinator.com/item?id=32585950 https://news.ycombinator.com/item?id=32585950

rcarmo 4y ago

I really want this to work on AMD hardware.

ArcVRArthur 4y ago
I hope I can work with them some time to improve support! I outlined a few things they would need to do to help support GVM using the amdgpu driver on this page: https://openmdev.io/index.php/AMDGPU https://openmdev.io/index.php/AMDGPU Ideally some folks who know about amdgpu might consider helping our open source community by adding similar information to that page to the information we added on the Nvidia Open Kernel Modules page: https://openmdev.io/index.php/OpenRM https://openmdev.io/index.php/OpenRM If that could be done then we would do our best to add in AMD support to GVM.

montecarl 4y ago

I run Linux for software development, but I need to also run Autodesk Fusion 360, which doesn't run on Linux. So I run it in a VirtualBox virtual machine. It works poorly as it sometimes doesn't render correctly and the performance is bad. I know that if I did some research and bought a second GPU that I could maybe get PCI-passthrough working, but I find that obnoxious and wasteful. Can I use LibVF.IO/GVM to get better performance (and perhaps proper rendering) on a VM in Linux? Or is there a different solution I should be looking into?

ArcVRArthur 4y ago
Ya, LibVF.IO & GVM are built for things like this! For example I have a friend who uses it for various Adobe programs which also don't work well on Linux.
amarshall 4y ago
It depends if your GPU is supported, and such support is mostly beholden to the hardware before anything else. Currently it’s largely limited to: for Nvidia, most Maxwell, Pascal, & Turing cards; for AMD, very few obsolete cards; for Intel, gen 6–9 iGPUs.
- ArcVRArthur 4y ago
  We have Ampere support working on some devices (and it is in development on other Ampere devices) as well as 11th & 12th generation Intel Xe. :) Here's the GPU Support page if you'd like to take a look: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support
  - Scramblejams 4y ago
    Love this project. I nerded out on GPU passthrough several years ago and learned a lot. GPU sharing would be a killer feature for some of my workflows. Recently, after being a long-time Nvidia customer, for Linux systems I've switched to AMD GPUs and have been very impressed with the amdgpu driver quality relative to Nvidia's proprietary driver. Do you have any thoughts on what the LibVF/GVM future looks like for AMD devices?

deleted 4y ago

[deleted]

Teever 4y ago

Can I use this with Proxmox? I looked into LibVF.IO when it was last posted on hackernews but I was never able to get it to work with Proxmox. I'm kind of a weirdo and want to use proxmox as a hypervisor on my desktop and manjaro as my daily driver with Windows 10 for Fusion360.

ajvs 4y ago

What's the recommended GPUs with best performance:cost?

ArcVRArthur 4y ago
That depends on your use-case. In general I would recommend you consider purchasing Nvidia's GPUs for the best price/performance and GVM support. Intel's Xe architecture is currently improving but the performance isn't quite there for a number of use-cases however some appear to work quite well and I expect that will improve with time. The 2080Ti works well with current software. If you are a developer and would like to help us improve support for devices you can purchase a 3090Ti (support in GVM for this device is under active development).

deleted 4y ago

[deleted]