20 ms·

Show HN: vGPU and SR-IOV on consumer GPUs

203 points by ArcVRArthur 5y ago

tfcata 5y ago
Does this work on the Radeon 580 series? I have a 2070 but I'm reluctant to install nVidia's driver...
liuliu 5y ago
Great! Still requires vGPU support and the merged driver approach last time I tried won't support CUDA on host (I was probably the first one tried the merged driver thing with vgpu_unlocked?). Looking forward someone write a Vulkan driver on Windows that just shuttles down to the Linux host. virgl used to be a promising project ...
- ArcVRArthur 5y ago
  I CUDA does work on the host in our testing. :) We also run Vulkan/DirectX/OpenGL at full performance in the guest! It's WAY faster than Virgl.
ArtWomb 5y ago
Whoa! Multi GPU is so hard. This could be the start of something. Looks like it makes use of nvidia's capture api?
- ArcVRArthur 5y ago
  We've been working on this for awhile and ya, I hope it is the start of something! If you want to read the logical origin point of where I started thinking about this project check this out: https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ https://arccompute.com/blog/why-computers-suck-and-how-openb... https://news.ycombinator.com/item?id=21776370 https://news.ycombinator.com/item?id=21776370 We're not using Nvidia's APIs in our code! libvf.io is entirely based on cross-vendor, standard APIs. :D The one that is a bit weird is AMD who is fairly far behind Nvidia and Intel but I think if they want to modernize support they can definitely do it. My hope is that consumers will start asking for this capability more (especially with Intel's ARC GPUs enabling GVT-g at every price point) and AMD will update their stuff to reflect that demand.
ncmncm 5y ago
They had me until YAML.
- anentropic 5y ago
  yes, but interestingly it seems to be https://nimyaml.org/ https://nimyaml.org/ :)
kfprt 5y ago
Note Intel GVT-g is only available on <=Gen9 GPU's. Gen11 and 12 are not supported.
- orangepurple 5y ago
  GVT-G has hard resolution limitations and was never stable with QEMU on Linux
  - ArcVRArthur 5y ago
    GVT-g uses KVMgt. Intel has not yet upstreamed their patches.
- ArcVRArthur 5y ago
  It also works on Intel's ARC and DG1/DG2 discrete GPUs. Our software also works on most consumer Nvidia cards and also some consumer AMD cards.
  - kfprt 5y ago
    Do you have any specific knowledge of Intels support for graphics virtualization on ARC GPU's? What are the security implications of LibVF and would it be compatible with the QubesOS security model?
    - ArcVRArthur 5y ago
      Ya, they are enabling GVT-g on all their GPUs going forward including the ARC branded devices (not just the embedded GPUs with Xe branding). They have no plans to lock out those features on consumer devices. In fact I bought a $700 computer from Best Buy that had an intel discrete GPU in it for development. :) Ya, I have been talking with the Qubes guys. I'm trying my best to figure out how I can help them ship something with LibVF.IO but they are currently based on Xen which uses Libxl rather than VFIO which may present some challenges.
      - 5y ago
gh123man 5y ago
This is insanely impressive. Having tried set up GPU passthrough in proxmox a few years ago, it was an absolute disaster. I would love to see this kind of approach more widely supported by other hypervisors! It's a real shame consumer GPUs are arbitrary locked down when the enterprise counterparts (often with the exact same chip) have much better support for virtualization.
- kfprt 5y ago
  The enterprise versions of these products have a lot of bugs and gotchas because so few use the feature.
  - ArcVRArthur 5y ago
    I'm making this because I had the same problem. My hope is that we can move the needle a little bit.
    - kfprt 5y ago
      As far as I'm concerned anyone that's working of virtualized GPU is doing the Lord's work.
- ArcVRArthur 5y ago
  Ya the lock out is absolutely arbitrary. There is zero physical difference between the consumer and server chips for these features. I think actually there's a lot of benefit to consumers by having these features enabled! I talk about that a bit here in our Xorg Developer Conference 2021 talk: https://www.youtube.com/watch?v=8pVrTyLqV_I https://www.youtube.com/watch?v=8pVrTyLqV_I We're going to try to add support for more distributions in the coming days. Right now we've got support in our install script for Ubuntu 20.04 hosts and arbitrary guest operating systems (Windows guests work best so far) but if people on GitHub are posting issues asking for support for other systems I'll try my best to get to those. I'm going to try to add official support for Arch, PopOS, and Fedora as I know some people who I think would use it on those systems and a few others.
uberduper 5y ago
I'm familiar with linux virtualization, gpu passthrough, etc. I've never heard of arcd and they've made no attempt in this doc or on their git to explain what it is or why it exists as, I assume, a replacement (or wrapper?) for qemu. My past experience with looking-glass is that it falls on its face at anything > 1440p@60Hz. I'm interested in vGPU for my linux VMs (spice is slow and sdl/gtk display is flakey) but for gaming, I don't want looking-glass and prefer to just do the passthrough thing with a KVM switch.
- ArcVRArthur 5y ago
  We are still trying to make the documentation better. We're actually hiring right now so if you can recommend someone who might be able to help with that I'd love to talk to them. :) Also looking-glass is an option under introspection: in the yaml. You can use whatever type of virtual display you like best. We chose looking glass as the default because in our testing it was the most performant.
- xt00 5y ago
  Yea what / where is arcd? is that a closed source portion of this stack that this depends upon? not super clear what is really open source about this stuff -- this looks like a pile of scripts to run arcd? edit: ok thanks so its qemu-system-x86_64 Have you guys tried to run something like android in the vm on an arm host? or what are the limitations there?
  - ArcVRArthur 5y ago
    All the source code for arcd is here: https://github.com/Arc-Compute/libvf.io/tree/master/src https://github.com/Arc-Compute/libvf.io/tree/master/src :)
- ArcVRArthur 5y ago
  arcd is actually the part of the program that you call in your shell to use our library's functions. Arc Compute is the name of our company so arcd seemed like a reasonable name. We also considered calling it vfd but so far we've settled on arcd.
- 5y ago
COGlory 5y ago
Very impressive. You should also post this to the Level1Techs forum
- ArcVRArthur 5y ago
  Wendell tweeted about LibVF.IO and it made my whole month haha: https://twitter.com/tekwendell/status/1449054328766013440 https://twitter.com/tekwendell/status/1449054328766013440 Also I posted in the Level1Techs forum! https://forum.level1techs.com/t/libvf-io-a-commodity-gpu-multiplexing-tool-driven-by-vfio-yaml/ https://forum.level1techs.com/t/libvf-io-a-commodity-gpu-mul...
DiabloD3 5y ago
I tried doing this years ago, and never quite go it to work. Some of the software involved in that article simply didn't exist yet, and GPUs weren't shipping with SR-IOV support yet (instead, I did Intel iGPU for Linux fbcon, real AMD GPU fed directly to the Windows VM with PCI-E Passthrough). In the end, I bailed on that dream and moved the Linux install to its own smaller machine, and ran Windows bare on the big machine. The problem was, if the GPU locked up hard, and GPUs back then would not respond to PCI Device Reset, if it wasn't something that merely re-initializing it on VM restart would fix... I had to restart the entire machine, thus defeating the purpose of having Windows in the VM in the first place! All my long-lived processes now run on the stand-alone Linux machine, and anything that is free to explode runs on my Windows machine. Windows gets wonky? Restart, ssh back into my screen sessions, reopen the browser, restart a bunch of cloud slaved apps, tada.
- ArcVRArthur 5y ago
  I got into writing this actually because graphics sharing was such a bad experience that I would usually run a Windows host and then Linux guests in something like VirtualBox or VMWare Workstation. I always wanted a host with fault tolerant guests but GPU sharing features seemed to be locked to "enterprise GPUs" and it wouldn't work on my consumer GPU. This solves some of those problems I was having and lets me run Linux on my host with a full performance Windows guest (at least for me it does).
  - starfallg 5y ago
    As the Nvidia Linux driver doesn't support DSC (Display Stream Compression), a lot of people are having problems driving 4k/240hz and 8k/60hz displays with DP 1.4 and HDMI 2.1. Frustratingly this works fine using Windows. Any ideas on how to run Linux guests on a Windows host while sharing the GPU like libvf.io? This would allow us work around the DSC issue until Nvidia implements support.
awesnvadsome 5y ago
This seems awesome! I have a passthrough setup with a very old card and a much newer one for games in a windows vm, it'll be nice to look into getting this set up and I can reduce the power draw on my system which was causing some problems... Is this something that could work with/be integrated with libvirt for easy configuration? Itd be neat to set it up with my current install, although not at all a real problem.
- ArcVRArthur 5y ago
  We actually are a replacement for Libvirt that tries to simplify a few things. Take a look at LibVF.IO's user API. I tried to make it a bit more human friendly, like Docker: https://github.com/Arc-Compute/libvf.io/blob/master/example/nvidia-mdev.yaml https://github.com/Arc-Compute/libvf.io/blob/master/example/...
  - awesnvadsome 5y ago
    Neat! Certainly more readable than the massive xml file! Thanks for the hard work! E: The code generating the qemu commands is also quite readable, so I think migrating my install will be quite doable + If needs be i can add args manually.
    - ArcVRArthur 5y ago
      Ya exactly! Compatibility with QEMU commands right inside the yaml was a great addition by my co-author. I wrote an earlier version that lacked that functionality and I have to say the new version with that feature has proven really handy.
      - encryptluks2 5y ago
        I wish all programs would consider YAML config files. Imagine being able to configure Chromium, Firefox, Thunderbird, Windows.. would make automation such a breeze. Chromium and Firefox has policy files but they are all JSON. I'm not even going to talk about the disaster of Windows automation.
rektide 5y ago
Really hoping AMD eventually does the right thing here. Not that it particularly matters seeing as how decent AMD video cards have been unpurchaseable for 18 months now. Consumers should have the ability to use their hardware well too. Selling the same thing at 2X the price differentiating only on virtualization capabilities is not a moral path. > We remain hopeful that AMD will recognize forthcoming changes in GPU virtualization with the creation of open standards such as Auxiliary Domains (AUX Domains), Mdev (VFIO-Mdev developed by Nvidia, RedHat, and Intel), and Alternative Routing-ID Interpretation (ARI) especially in light of Intel's market entrance with their ARC line of GPUs supporting Intel Graphics Virtualization Technology (GVT-g). Really cool to hear there are a bunch of vGPU-related efforts underway! That's so great.
- ArcVRArthur 5y ago
  I have to pinch myself from time to time to remind myself I'm not dreaming. I honestly feel like I'm working on the coolest project in the world with some of the smartest people I've ever met. If you know anyone who wants to help us work on open source vGPU stuff please reach out to me at arthur@arccompute.com. We're hiring!!!
- encryptluks2 5y ago
  I am disappointed in AMD. I bought AMD for Linux but when it comes to vGPU they don't give a shit. Intel was the leaders who made this semi-mainstream.
  - ArcVRArthur 5y ago
    It is very disappointing for sure. I think the best thing we can do right now as users and developers is to make our voices heard and to ask AMD to support VFIO-Mdev on consumer GPUs. If enough people ask I think they will dedicate the energy to supporting these new open APIs as the other vendors now have. We have to be clear that we want this on consumer gear, not some artificially locked out server card where they have relegated these capabilities to with zero technical reason to do so.
omreaderhn 5y ago
This looks really nice. Does this work with Ampere GPUs?
- ArcVRArthur 5y ago
  We're supporting Ampere in our own VPS cloud (arccompute.com) but there are some hurtles on getting it working on consumer Ampere gear in the nv merged driver due to changes in the use of SR-IOV APIs - that is to say Nvidia has now enabled SR-IOV whereas before they were doing their entirely different thing to achieve similar functionality. We've got SR-IOV APIs working with our use of mdevType: "sriovdev" in our yaml configuration layer so from our perspective it's already possible to use but unfortunately until the nv merged drivers work with Ampere we won't be able to do a lot on our end.. Right now I think the vGPU_Unlock guys are working on the 3090 and we'll be sure to support it once it's ready. I'll also try to remember to post about it on GitHub once I've tested it out for myself and confirmed our software works with it. By the way, we're hiring GPU driver engineers and kernel hackers! Please reach out to me if you know anyone who might be interested. My email is: arthur@arccompute.com
  - rirze 5y ago
    Super thrilled to hear you guys working on the 3090! I'll be keeping an eye out for any progress
deleted 5y ago
[deleted]
belval 5y ago
Just to clarify because I've failed at doing pretty much the setup that you are describing with my 1080Ti. Does this still require the vgpu_unlock changes for Nvidia cards or is this something that bypasses the need for it entirely?
- ArcVRArthur 5y ago
  My main development rig actually uses a 1080Ti! The normal vGPU_Unlock steps aren't a part of the user setup process, rather the optional merged driver package is built with a C version of the code that's been prepackaged. If you decide to use the vGPU_Unlock merged driver option during setup you won't have to do the (somewhat intense) process of vGPU_Unlock setup, you'll just skip straight to the end result. :)
  - kobalsky 5y ago
    Why is it optional? Why would someone want the option to use it if it’s not required? Sorry I don’t understand well. A lot of jargon is being throw around and I haven’t touched this topic much once I got my pass through setup working
csdvrx 5y ago
That's very impressive! You may want to do the same for NVMe: creating several namespaces is not supported on most consumer drives, while laptops can rarely have more than 1 NVMe (same problem as with the GPUs: a passthrough requires having 2 of them) Being able to split the NVMe drive not by partition but by namespace would let each OS see a "full drive".
- ArcVRArthur 5y ago
  That's actually a great idea! Is there any documentation you would suggest I read to get started understanding NVMe namespaces? We're also very open to feature additions/pull requests at our repo: https://libvf.io/ https://libvf.io/ I'm very new to this whole thing of building an open source community but I hope if people find value in some of what they built that they might consider helping us with our code. If you have any suggestions on how we could improve and build a good open source community I'd love to hear them!
  - csdvrx 5y ago
    > That's actually a great idea! Thanks! Besides being technically interesting and similar to what you do, I think it would also be quite useful for your target audience: if they don't want to do a GPU passthrough, maybe it's because they only have 1 GPU. Maybe they also have only 1 physical drive! > Is there any documentation you would suggest I read to get started understanding NVMe namespaces? Just check the official NVMe specs. You can also grab a drive that supports namespaces to see how it works in practice. It's not very complicated: if you have only 1 NVMe that supports namespaces, and on which you create 3 namespaces, you will have /dev/nvme0n1 /dev/nvme0n2 and /dev/nvme0n3 which can each be partitioned. If you pass /dev/nvme0n2 to a VM1 and /dev/nvme0n3 to another VM, you can be sure that neither will be able to write on the other VM volume, or to the partitions used by the host. It's that simple! So don't waste too much time on the official specs: the goal for your driver would be just to recognize partitions of a special type, intercept them, and present them to the kernel as if they were a namespace (so they can be passed to virtual machines as if they were a namespace...) for people who have NVMe drives that don't support namespaces (the same thing could be used for regular SATA spinning drives and SSD, but the smaller IOPS compared to NVMe would make it less useful in practice) Under the hood, you will also want to implement logical block checks to enforce barriers (just like the IOMMU!) so that VM1 can't write blocks on the VM2 virtual drive, and vice versa. Now that I think about it, it should be quite simple as partitions have a start point and an end point : just check if the write on the virtual disk offset will be within the partition boundaries once you add the offset to map that to the physical disk: for example if /dev/nvme0n2 corresponds to /dev/nvme0n1p6 the offset would simply be the start of this partition. Now, how to do that nicely, in a way that wouldn't confuse the host or cause problems when dual booting? A simple way to do that could be to use a well-known GUID. A good candidate would be 024DEE41-33E7-11D3-9D69-0008C781F39F which was reserved for "MBR partition scheme" but remains mostly unused: a partition under this GUID is expected to contain a full disk image (with partitions etc) but will be ignored: https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIVE-MBR https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIV... On the user side, when not using your driver, each 024DEE41-33E7-11D3-9D69-0008C781F39F type partition could be mounted with losetup -P to read the partition table: https://stackoverflow.com/questions/37227233/having-losetup-read-the-partition-table https://stackoverflow.com/questions/37227233/having-losetup-... After thinking a little more about how it would work, it should be a quite interesting project! I hope you will do that! > We're also very open to feature additions/pull requests Unfortunately, I don't have a lot of time for such projects at the moment but that should be enough to get you started. > If you have any suggestions on how we could improve and build a good open source community I'd love to hear them! Release early, release often, even if it's a bare-mininum MVP. Announce it here. And shoot me an email at my username at outlook.com so I can do the same in other places :)
AaronFriel 5y ago
This article almost got my hopes up too much! I'm curious whether Intel's "Ark" GPUs will support the same or if they'll go the same path of Nvidia in locking down virtual function support.
- ArcVRArthur 5y ago
  They have made the APIs open on all of their discrete GPUs going forward. I actually have an Intel discrete GPU board (based on the Intel's DG1 - Discrete Graphics 1) that supports GVT-g and all the same standard APIs we deal with. I think Intel is going to be a huge disruptor in the GPU market because of this! The good news is that we also support consumer Nvidia GPUs and some consumer AMD GPUs as well. There are more limitations in AMD land than the other vendors but I'm hopeful that they will make some improvements across time.
shmerl 5y ago
What's the story with SR-IOV for consumer AMD GPUs on Linux? Last time I looked into it, it was impossible to use or more like AMD didn't want to support it.
danbmil99 5y ago
Very cool approach. It's going to be a fight I suspect; vendors lock things up specifically so they can have price differentials in different markets. It may end up being like the fight between workstations versus PCS back in the 90s
- ArcVRArthur 5y ago
  I actually think that Intel GVT-g has opened the vGPU flood gates. With Intel trying to differentiate themselves as a new market participant they've just decided to leave this functionality on for all the parts. I bought a 700$ prebuilt PC from Best Buy with an Intel DG1 GPU and it's got GVT-g. If you look at the kernel API for this stuff you'll notice RedHat, Nvidia, and Intel wrote it together. I think Nvidia is just adapting to what will be the new market norm as the Intel GPUs proliferate. Things are becoming more open, not less! That's partly why I wanted to make this free open source software, to help people take advantage of these cool new open source kernel capabilities on consumer gear!
scarygliders 5y ago
Very, very cool! I've set up a dual-GPU system in the past using two nvidia GPUs and whilst I found the trek towards PCI passthrough to other virtual machines rewarding when it finally worked, I also found the arrangement to be inconvenient. What you've achieved here, seems the ideal. Well done :) I will either patiently wait for an Arch Linux version of the install, or I'll eventually end up impatient and see if I can rustle up something - an install script is an install script, it should be just a matter (famous last words) of altering the install script/procedures to suit.
compsciphd 5y ago
isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into thinking that the consumer gpu is really the server model, but otherwise it becomes just normal nvidia usage?
- ArcVRArthur 5y ago
  Ya, libvf.io works on nearly all recent Nvidia consumer cards with the exception of Ampere. The same goes for Intel GPUs like the DG1/DG2. There is no need to use a server card. Edit: There's not really an explicit need to trick the card. Nvidia co-wrote the VFIO-Mdev API with Intel and RedHat which is what we use. If you want to use the optional nv merged driver package that will do some of the things you mention but it's not required.
- zamadatix 5y ago
  It's natively supported by all of the GPUs listed, the trick is indeed bringing this to the consumer space for the first time as it has always been model locked and sometimes (e.g. Nvidia) license locked. The exception to these restrictions is with the Intel method.
  - ArcVRArthur 5y ago
    Nvidia co-wrote the VFIO-Mdev API in partnership with Intel where it is used as a kernel interface for GVT-g mediated GPU devices (as well as Nvidia mediated devices). My (totally baseless) guess would be that Nvidia saw this market change coming with Intel ARC GPUs leaving vGPU functionality enabled across the consumer lineup and rather than getting left behind by a big change they helped bring it about. We're just exposing the capabilities of that API to the user across vendors and making it easy to interface with.
- remless 5y ago
  Yes the functionality is available on nvidia quadro cards and nvidia very recently added the same options to some of their more recent consumer cards specifically for HyperV. I will say though from a hobbyist point of view this project is just great. If you're trying things and spinning up VMs just to see what you can do then one of the first bottle necks is going to be that you can't use dedicated graphics (without passing through the whole card, and then you need a card per VM). This project makes it a lot easier for anyone fiddling rather than doing stuff commercially, especially with current GPU prices.
amelius 5y ago
Does this also work for CUDA (e.g. deep learning jobs)?
- ArcVRArthur 5y ago
  CUDA works on the host. Stay tuned for more info about guest CUDA from libvf.io ;)
Kaze404 5y ago
Once this is running on NixOS I'll be all over it. Great news!
- ArcVRArthur 5y ago
  There's a guy in our Matrix and Discord channels that's working on a Nix package. :) I'll introduce you to him if you'd like.

203 points by ArcVRArthur 5y ago

Does this work on the Radeon 580 series? I have a 2070 but I'm reluctant to install nVidia's driver...

Great! Still requires vGPU support and the merged driver approach last time I tried won't support CUDA on host (I was probably the first one tried the merged driver thing with vgpu_unlocked?). Looking forward someone write a Vulkan driver on Windows that just shuttles down to the Linux host. virgl used to be a promising project ...

ArcVRArthur 5y ago
I CUDA does work on the host in our testing. :) We also run Vulkan/DirectX/OpenGL at full performance in the guest! It's WAY faster than Virgl.

ArtWomb 5y ago

Whoa! Multi GPU is so hard. This could be the start of something. Looks like it makes use of nvidia's capture api?

ArcVRArthur 5y ago
We've been working on this for awhile and ya, I hope it is the start of something! If you want to read the logical origin point of where I started thinking about this project check this out: https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ https://arccompute.com/blog/why-computers-suck-and-how-openb... https://news.ycombinator.com/item?id=21776370 https://news.ycombinator.com/item?id=21776370 We're not using Nvidia's APIs in our code! libvf.io is entirely based on cross-vendor, standard APIs. :D The one that is a bit weird is AMD who is fairly far behind Nvidia and Intel but I think if they want to modernize support they can definitely do it. My hope is that consumers will start asking for this capability more (especially with Intel's ARC GPUs enabling GVT-g at every price point) and AMD will update their stuff to reflect that demand.

ncmncm 5y ago

They had me until YAML.

anentropic 5y ago
yes, but interestingly it seems to be https://nimyaml.org/ https://nimyaml.org/ :)

kfprt 5y ago

Note Intel GVT-g is only available on <=Gen9 GPU's. Gen11 and 12 are not supported.

orangepurple 5y ago
GVT-G has hard resolution limitations and was never stable with QEMU on Linux
- ArcVRArthur 5y ago
  GVT-g uses KVMgt. Intel has not yet upstreamed their patches.
ArcVRArthur 5y ago
It also works on Intel's ARC and DG1/DG2 discrete GPUs. Our software also works on most consumer Nvidia cards and also some consumer AMD cards.
- kfprt 5y ago
  Do you have any specific knowledge of Intels support for graphics virtualization on ARC GPU's? What are the security implications of LibVF and would it be compatible with the QubesOS security model?
  - ArcVRArthur 5y ago
    Ya, they are enabling GVT-g on all their GPUs going forward including the ARC branded devices (not just the embedded GPUs with Xe branding). They have no plans to lock out those features on consumer devices. In fact I bought a $700 computer from Best Buy that had an intel discrete GPU in it for development. :) Ya, I have been talking with the Qubes guys. I'm trying my best to figure out how I can help them ship something with LibVF.IO but they are currently based on Xen which uses Libxl rather than VFIO which may present some challenges.
    - 5y ago

gh123man 5y ago

This is insanely impressive. Having tried set up GPU passthrough in proxmox a few years ago, it was an absolute disaster. I would love to see this kind of approach more widely supported by other hypervisors! It's a real shame consumer GPUs are arbitrary locked down when the enterprise counterparts (often with the exact same chip) have much better support for virtualization.

kfprt 5y ago
The enterprise versions of these products have a lot of bugs and gotchas because so few use the feature.
- ArcVRArthur 5y ago
  I'm making this because I had the same problem. My hope is that we can move the needle a little bit.
  - kfprt 5y ago
    As far as I'm concerned anyone that's working of virtualized GPU is doing the Lord's work.
ArcVRArthur 5y ago
Ya the lock out is absolutely arbitrary. There is zero physical difference between the consumer and server chips for these features. I think actually there's a lot of benefit to consumers by having these features enabled! I talk about that a bit here in our Xorg Developer Conference 2021 talk: https://www.youtube.com/watch?v=8pVrTyLqV_I https://www.youtube.com/watch?v=8pVrTyLqV_I We're going to try to add support for more distributions in the coming days. Right now we've got support in our install script for Ubuntu 20.04 hosts and arbitrary guest operating systems (Windows guests work best so far) but if people on GitHub are posting issues asking for support for other systems I'll try my best to get to those. I'm going to try to add official support for Arch, PopOS, and Fedora as I know some people who I think would use it on those systems and a few others.

uberduper 5y ago

I'm familiar with linux virtualization, gpu passthrough, etc. I've never heard of arcd and they've made no attempt in this doc or on their git to explain what it is or why it exists as, I assume, a replacement (or wrapper?) for qemu. My past experience with looking-glass is that it falls on its face at anything > 1440p@60Hz. I'm interested in vGPU for my linux VMs (spice is slow and sdl/gtk display is flakey) but for gaming, I don't want looking-glass and prefer to just do the passthrough thing with a KVM switch.

ArcVRArthur 5y ago
We are still trying to make the documentation better. We're actually hiring right now so if you can recommend someone who might be able to help with that I'd love to talk to them. :) Also looking-glass is an option under introspection: in the yaml. You can use whatever type of virtual display you like best. We chose looking glass as the default because in our testing it was the most performant.
xt00 5y ago
Yea what / where is arcd? is that a closed source portion of this stack that this depends upon? not super clear what is really open source about this stuff -- this looks like a pile of scripts to run arcd? edit: ok thanks so its qemu-system-x86_64 Have you guys tried to run something like android in the vm on an arm host? or what are the limitations there?
- ArcVRArthur 5y ago
  All the source code for arcd is here: https://github.com/Arc-Compute/libvf.io/tree/master/src https://github.com/Arc-Compute/libvf.io/tree/master/src :)
ArcVRArthur 5y ago
arcd is actually the part of the program that you call in your shell to use our library's functions. Arc Compute is the name of our company so arcd seemed like a reasonable name. We also considered calling it vfd but so far we've settled on arcd.
5y ago

COGlory 5y ago

Very impressive. You should also post this to the Level1Techs forum

ArcVRArthur 5y ago
Wendell tweeted about LibVF.IO and it made my whole month haha: https://twitter.com/tekwendell/status/1449054328766013440 https://twitter.com/tekwendell/status/1449054328766013440 Also I posted in the Level1Techs forum! https://forum.level1techs.com/t/libvf-io-a-commodity-gpu-multiplexing-tool-driven-by-vfio-yaml/ https://forum.level1techs.com/t/libvf-io-a-commodity-gpu-mul...

DiabloD3 5y ago

I tried doing this years ago, and never quite go it to work. Some of the software involved in that article simply didn't exist yet, and GPUs weren't shipping with SR-IOV support yet (instead, I did Intel iGPU for Linux fbcon, real AMD GPU fed directly to the Windows VM with PCI-E Passthrough). In the end, I bailed on that dream and moved the Linux install to its own smaller machine, and ran Windows bare on the big machine. The problem was, if the GPU locked up hard, and GPUs back then would not respond to PCI Device Reset, if it wasn't something that merely re-initializing it on VM restart would fix... I had to restart the entire machine, thus defeating the purpose of having Windows in the VM in the first place! All my long-lived processes now run on the stand-alone Linux machine, and anything that is free to explode runs on my Windows machine. Windows gets wonky? Restart, ssh back into my screen sessions, reopen the browser, restart a bunch of cloud slaved apps, tada.

ArcVRArthur 5y ago
I got into writing this actually because graphics sharing was such a bad experience that I would usually run a Windows host and then Linux guests in something like VirtualBox or VMWare Workstation. I always wanted a host with fault tolerant guests but GPU sharing features seemed to be locked to "enterprise GPUs" and it wouldn't work on my consumer GPU. This solves some of those problems I was having and lets me run Linux on my host with a full performance Windows guest (at least for me it does).
- starfallg 5y ago
  As the Nvidia Linux driver doesn't support DSC (Display Stream Compression), a lot of people are having problems driving 4k/240hz and 8k/60hz displays with DP 1.4 and HDMI 2.1. Frustratingly this works fine using Windows. Any ideas on how to run Linux guests on a Windows host while sharing the GPU like libvf.io? This would allow us work around the DSC issue until Nvidia implements support.

awesnvadsome 5y ago

This seems awesome! I have a passthrough setup with a very old card and a much newer one for games in a windows vm, it'll be nice to look into getting this set up and I can reduce the power draw on my system which was causing some problems... Is this something that could work with/be integrated with libvirt for easy configuration? Itd be neat to set it up with my current install, although not at all a real problem.

ArcVRArthur 5y ago
We actually are a replacement for Libvirt that tries to simplify a few things. Take a look at LibVF.IO's user API. I tried to make it a bit more human friendly, like Docker: https://github.com/Arc-Compute/libvf.io/blob/master/example/nvidia-mdev.yaml https://github.com/Arc-Compute/libvf.io/blob/master/example/...
- awesnvadsome 5y ago
  Neat! Certainly more readable than the massive xml file! Thanks for the hard work! E: The code generating the qemu commands is also quite readable, so I think migrating my install will be quite doable + If needs be i can add args manually.
  - ArcVRArthur 5y ago
    Ya exactly! Compatibility with QEMU commands right inside the yaml was a great addition by my co-author. I wrote an earlier version that lacked that functionality and I have to say the new version with that feature has proven really handy.
    - encryptluks2 5y ago
      I wish all programs would consider YAML config files. Imagine being able to configure Chromium, Firefox, Thunderbird, Windows.. would make automation such a breeze. Chromium and Firefox has policy files but they are all JSON. I'm not even going to talk about the disaster of Windows automation.

rektide 5y ago

Really hoping AMD eventually does the right thing here. Not that it particularly matters seeing as how decent AMD video cards have been unpurchaseable for 18 months now. Consumers should have the ability to use their hardware well too. Selling the same thing at 2X the price differentiating only on virtualization capabilities is not a moral path. > We remain hopeful that AMD will recognize forthcoming changes in GPU virtualization with the creation of open standards such as Auxiliary Domains (AUX Domains), Mdev (VFIO-Mdev developed by Nvidia, RedHat, and Intel), and Alternative Routing-ID Interpretation (ARI) especially in light of Intel's market entrance with their ARC line of GPUs supporting Intel Graphics Virtualization Technology (GVT-g). Really cool to hear there are a bunch of vGPU-related efforts underway! That's so great.

ArcVRArthur 5y ago
I have to pinch myself from time to time to remind myself I'm not dreaming. I honestly feel like I'm working on the coolest project in the world with some of the smartest people I've ever met. If you know anyone who wants to help us work on open source vGPU stuff please reach out to me at arthur@arccompute.com. We're hiring!!!
encryptluks2 5y ago
I am disappointed in AMD. I bought AMD for Linux but when it comes to vGPU they don't give a shit. Intel was the leaders who made this semi-mainstream.
- ArcVRArthur 5y ago
  It is very disappointing for sure. I think the best thing we can do right now as users and developers is to make our voices heard and to ask AMD to support VFIO-Mdev on consumer GPUs. If enough people ask I think they will dedicate the energy to supporting these new open APIs as the other vendors now have. We have to be clear that we want this on consumer gear, not some artificially locked out server card where they have relegated these capabilities to with zero technical reason to do so.

omreaderhn 5y ago

This looks really nice. Does this work with Ampere GPUs?

ArcVRArthur 5y ago
We're supporting Ampere in our own VPS cloud (arccompute.com) but there are some hurtles on getting it working on consumer Ampere gear in the nv merged driver due to changes in the use of SR-IOV APIs - that is to say Nvidia has now enabled SR-IOV whereas before they were doing their entirely different thing to achieve similar functionality. We've got SR-IOV APIs working with our use of mdevType: "sriovdev" in our yaml configuration layer so from our perspective it's already possible to use but unfortunately until the nv merged drivers work with Ampere we won't be able to do a lot on our end.. Right now I think the vGPU_Unlock guys are working on the 3090 and we'll be sure to support it once it's ready. I'll also try to remember to post about it on GitHub once I've tested it out for myself and confirmed our software works with it. By the way, we're hiring GPU driver engineers and kernel hackers! Please reach out to me if you know anyone who might be interested. My email is: arthur@arccompute.com
- rirze 5y ago
  Super thrilled to hear you guys working on the 3090! I'll be keeping an eye out for any progress

deleted 5y ago

[deleted]

belval 5y ago

Just to clarify because I've failed at doing pretty much the setup that you are describing with my 1080Ti. Does this still require the vgpu_unlock changes for Nvidia cards or is this something that bypasses the need for it entirely?

ArcVRArthur 5y ago
My main development rig actually uses a 1080Ti! The normal vGPU_Unlock steps aren't a part of the user setup process, rather the optional merged driver package is built with a C version of the code that's been prepackaged. If you decide to use the vGPU_Unlock merged driver option during setup you won't have to do the (somewhat intense) process of vGPU_Unlock setup, you'll just skip straight to the end result. :)
- kobalsky 5y ago
  Why is it optional? Why would someone want the option to use it if it’s not required? Sorry I don’t understand well. A lot of jargon is being throw around and I haven’t touched this topic much once I got my pass through setup working

csdvrx 5y ago

That's very impressive! You may want to do the same for NVMe: creating several namespaces is not supported on most consumer drives, while laptops can rarely have more than 1 NVMe (same problem as with the GPUs: a passthrough requires having 2 of them) Being able to split the NVMe drive not by partition but by namespace would let each OS see a "full drive".

ArcVRArthur 5y ago
That's actually a great idea! Is there any documentation you would suggest I read to get started understanding NVMe namespaces? We're also very open to feature additions/pull requests at our repo: https://libvf.io/ https://libvf.io/ I'm very new to this whole thing of building an open source community but I hope if people find value in some of what they built that they might consider helping us with our code. If you have any suggestions on how we could improve and build a good open source community I'd love to hear them!
- csdvrx 5y ago
  > That's actually a great idea! Thanks! Besides being technically interesting and similar to what you do, I think it would also be quite useful for your target audience: if they don't want to do a GPU passthrough, maybe it's because they only have 1 GPU. Maybe they also have only 1 physical drive! > Is there any documentation you would suggest I read to get started understanding NVMe namespaces? Just check the official NVMe specs. You can also grab a drive that supports namespaces to see how it works in practice. It's not very complicated: if you have only 1 NVMe that supports namespaces, and on which you create 3 namespaces, you will have /dev/nvme0n1 /dev/nvme0n2 and /dev/nvme0n3 which can each be partitioned. If you pass /dev/nvme0n2 to a VM1 and /dev/nvme0n3 to another VM, you can be sure that neither will be able to write on the other VM volume, or to the partitions used by the host. It's that simple! So don't waste too much time on the official specs: the goal for your driver would be just to recognize partitions of a special type, intercept them, and present them to the kernel as if they were a namespace (so they can be passed to virtual machines as if they were a namespace...) for people who have NVMe drives that don't support namespaces (the same thing could be used for regular SATA spinning drives and SSD, but the smaller IOPS compared to NVMe would make it less useful in practice) Under the hood, you will also want to implement logical block checks to enforce barriers (just like the IOMMU!) so that VM1 can't write blocks on the VM2 virtual drive, and vice versa. Now that I think about it, it should be quite simple as partitions have a start point and an end point : just check if the write on the virtual disk offset will be within the partition boundaries once you add the offset to map that to the physical disk: for example if /dev/nvme0n2 corresponds to /dev/nvme0n1p6 the offset would simply be the start of this partition. Now, how to do that nicely, in a way that wouldn't confuse the host or cause problems when dual booting? A simple way to do that could be to use a well-known GUID. A good candidate would be 024DEE41-33E7-11D3-9D69-0008C781F39F which was reserved for "MBR partition scheme" but remains mostly unused: a partition under this GUID is expected to contain a full disk image (with partitions etc) but will be ignored: https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIVE-MBR https://en.wikipedia.org/wiki/GUID_Partition_Table#PROTECTIV... On the user side, when not using your driver, each 024DEE41-33E7-11D3-9D69-0008C781F39F type partition could be mounted with losetup -P to read the partition table: https://stackoverflow.com/questions/37227233/having-losetup-read-the-partition-table https://stackoverflow.com/questions/37227233/having-losetup-... After thinking a little more about how it would work, it should be a quite interesting project! I hope you will do that! > We're also very open to feature additions/pull requests Unfortunately, I don't have a lot of time for such projects at the moment but that should be enough to get you started. > If you have any suggestions on how we could improve and build a good open source community I'd love to hear them! Release early, release often, even if it's a bare-mininum MVP. Announce it here. And shoot me an email at my username at outlook.com so I can do the same in other places :)

AaronFriel 5y ago

This article almost got my hopes up too much! I'm curious whether Intel's "Ark" GPUs will support the same or if they'll go the same path of Nvidia in locking down virtual function support.

ArcVRArthur 5y ago
They have made the APIs open on all of their discrete GPUs going forward. I actually have an Intel discrete GPU board (based on the Intel's DG1 - Discrete Graphics 1) that supports GVT-g and all the same standard APIs we deal with. I think Intel is going to be a huge disruptor in the GPU market because of this! The good news is that we also support consumer Nvidia GPUs and some consumer AMD GPUs as well. There are more limitations in AMD land than the other vendors but I'm hopeful that they will make some improvements across time.

shmerl 5y ago

What's the story with SR-IOV for consumer AMD GPUs on Linux? Last time I looked into it, it was impossible to use or more like AMD didn't want to support it.

danbmil99 5y ago

Very cool approach. It's going to be a fight I suspect; vendors lock things up specifically so they can have price differentials in different markets. It may end up being like the fight between workstations versus PCS back in the 90s

ArcVRArthur 5y ago
I actually think that Intel GVT-g has opened the vGPU flood gates. With Intel trying to differentiate themselves as a new market participant they've just decided to leave this functionality on for all the parts. I bought a 700$ prebuilt PC from Best Buy with an Intel DG1 GPU and it's got GVT-g. If you look at the kernel API for this stuff you'll notice RedHat, Nvidia, and Intel wrote it together. I think Nvidia is just adapting to what will be the new market norm as the Intel GPUs proliferate. Things are becoming more open, not less! That's partly why I wanted to make this free open source software, to help people take advantage of these cool new open source kernel capabilities on consumer gear!

scarygliders 5y ago

Very, very cool! I've set up a dual-GPU system in the past using two nvidia GPUs and whilst I found the trek towards PCI passthrough to other virtual machines rewarding when it finally worked, I also found the arrangement to be inconvenient. What you've achieved here, seems the ideal. Well done :) I will either patiently wait for an Arch Linux version of the install, or I'll eventually end up impatient and see if I can rustle up something - an install script is an install script, it should be just a matter (famous last words) of altering the install script/procedures to suit.

compsciphd 5y ago

isn't this natively supported by nvidia? i.e. you have a vgpu card (or a consumer card you an map to the equivalent vgpu card) and nvidia drivers and tools lets you load drivers that essetially partition it into XGB (all partitions being the same X) and then you just gpu passthrough the newly created device that maps to a single partition into the vm? the whole trick being the ability to trick nvidia's drivers into thinking that the consumer gpu is really the server model, but otherwise it becomes just normal nvidia usage?

ArcVRArthur 5y ago
Ya, libvf.io works on nearly all recent Nvidia consumer cards with the exception of Ampere. The same goes for Intel GPUs like the DG1/DG2. There is no need to use a server card. Edit: There's not really an explicit need to trick the card. Nvidia co-wrote the VFIO-Mdev API with Intel and RedHat which is what we use. If you want to use the optional nv merged driver package that will do some of the things you mention but it's not required.
zamadatix 5y ago
It's natively supported by all of the GPUs listed, the trick is indeed bringing this to the consumer space for the first time as it has always been model locked and sometimes (e.g. Nvidia) license locked. The exception to these restrictions is with the Intel method.
- ArcVRArthur 5y ago
  Nvidia co-wrote the VFIO-Mdev API in partnership with Intel where it is used as a kernel interface for GVT-g mediated GPU devices (as well as Nvidia mediated devices). My (totally baseless) guess would be that Nvidia saw this market change coming with Intel ARC GPUs leaving vGPU functionality enabled across the consumer lineup and rather than getting left behind by a big change they helped bring it about. We're just exposing the capabilities of that API to the user across vendors and making it easy to interface with.
remless 5y ago
Yes the functionality is available on nvidia quadro cards and nvidia very recently added the same options to some of their more recent consumer cards specifically for HyperV. I will say though from a hobbyist point of view this project is just great. If you're trying things and spinning up VMs just to see what you can do then one of the first bottle necks is going to be that you can't use dedicated graphics (without passing through the whole card, and then you need a card per VM). This project makes it a lot easier for anyone fiddling rather than doing stuff commercially, especially with current GPU prices.

amelius 5y ago

Does this also work for CUDA (e.g. deep learning jobs)?

ArcVRArthur 5y ago
CUDA works on the host. Stay tuned for more info about guest CUDA from libvf.io ;)

Kaze404 5y ago

Once this is running on NixOS I'll be all over it. Great news!

ArcVRArthur 5y ago
There's a guy in our Matrix and Discord channels that's working on a Nix package. :) I'll introduce you to him if you'd like.