> Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I

> Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I am happy to say though that Intel now uses SR-IOV instead of GVT-g (deprecated since 9th generation Intel). Their replacement SR-IOV driver is now Open Source (recently made public): This would legitimately be ideal information to have in the readme/top-level link. > https://github.com/intel/linux-intel-lts/commit/41ef979f0894 https://github.com/intel/linux-intel-lts/commit/41ef979f0894 This is pretty unhelpful. Legitimately. It's not mainlined yet, there are zero userspace docs, etc. The patch looks like it will pretty much "just work" when/if Intel bothers to get it into mainline. Until then, a patched/forked kernel is needed. > Also to address the first comment in this thread - there are many inaccuracies here: > Post-Ampere supports MIG and SR-IOV. VFIO-Mediated Devices (Mdev) are used both pre-Ampere and post-Ampere. This is how that works: > https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals I maintained mdev support for a major KVM-based platform, but it's been a couple of years. That said, a link to how mdev internals work isn't useful to end-users, who just want to know "how do I partition my card"? As-in "which driver/utilities do I need to install"? > For folks who are interested we also built LibVF.IO which enables vGPU/SR-IOV functionality on consumer GPUs: > https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 > If you're interested in a full list of supported GPUs you can read the following page from our wiki: > https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support Is there some way in which LibVF.IO differs from just being a wrapper around KVM/qemu? Because the scripts do an awful lot of stuff to your host system, and arcd.nim appears to just call qemu anyway: https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libvfio/control/vm.nim https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libv... Sure, it also binds/removes mdev devices, which is a nice convenience, and you have a couple of patches applied to the nvidia driver sources, but asking users to blindly execute scripts they have to wade through to find out exactly what they're going to do in kernelspace, plus the system. It's... asking a lot. You're adding a virtual sound card, nim, shell aliases, samba, plasma, then blindly overwriting any kernel options the user has set without even the good grace of capturing them and appending them: https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/funcs-libvfio.sh#L184 https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/... > Also finally this tool has nothing to do with nvidia-cli or mdevctl. It defines available mdev devices in the mdevctl list, it does not replace mdevctl. It looks like it does a lot more than that, and less than that. It has nothing to do with nvidia-cli (which can also manage mdev devices), and "defines available mdev devices in the mdevctl list" only as a side effect of the fact that it's doing a whole bunch of other stuff to the system. I'm not trying to tear down your project, but honestly, the docs could be far, far better about what this actually does, how it does it, which pieces you actually need, etc. Because it certainly looks like all of the system configuration could be done with ansible/terraform instead of shell and published as an associated project and/or prerequisite steps where users would be informed of what's happening to their system. Similarly, it strongly appears that arcd could be more or less replaced by any given binding to the libvirt API, which would also allow the VMs to be easily migrated (assuming identical hardware), the libvirt XML shared with others, snapshotting, storage pooling, listed in virt-manager/virsh, and so on. As a basic open source citizen, this would also help in "giving credit where credit is due". Bravo for stitching all of this together, but the project pages/repo certainly make it seem as if LibVF.IO/GVM did the work, and that's not really honest. GVM, rather than "a GPU Virtual Machine ..." is Haskell bindings for the RMAPI. LibVF.IO is "Utilities KVM+qemu VMs with vGPU passthrough for humans" If there's a mistake here, please correct it, but going through the repos, it looks like a whole lot of glue. There's nothing wrong with that. It's still valuable. It's just that all of the work you did in researching/testing/building it could (and arguably should) just as well go somewhere like where one of the original devs working on vfio/GPU passthrough posted a five page braindump of everything you'd ever need to know which people have been shamelessly cribbing from (on the Arch Wiki and others) for years, maybe without knowing: http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-1-hardware.html http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part... It would be extremely valuable to the community to document HOW end-users can tweak all of these knobs WITHOUT windmill slamming a bunch of packages from scripts in your repo and relying on your nim glue to do it.