4 ms·

> Pre-Ampere GPUs (Ampere+ is MIG) More complex than that. MIG covers compute use cases only. For workloads where graphics are needed (or even the graphics API

by my123 4y ago

> Pre-Ampere GPUs (Ampere+ is MIG) More complex than that. MIG covers compute use cases only. For workloads where graphics are needed (or even the graphics APIs), you have to use preemptive scheduling, even on Ampere. > which is reasonably widely supported on Gen12 and above No. It's gone on Gen11 and later. :/ And no replacement yet. > MxGPU is only supported on some FirePro cards Forget about AMD cards for GPU virtualisation. The modern GIM drivers aren't public. So you're stuck with very old out of support GPUs. > and it's also unclear whether this is tied to the (very much "don't use in production") open source nvidia drivers. The OSS NV KM stack doesn't support GPU virt at all yet.

evol262 4y ago
> More complex than that. MIG covers compute use cases only. For workloads where graphics are needed (or even the graphics APIs), you have to use preemptive scheduling, even on Ampere. Thanks for the correction! Shows how much I've really used it outside of "does nvidia-smi do the right thing, and can I map those to pods". I had assumed that, since nvidia-smi on Ampere did both GPU and CPU slicing, that maybe they found a solution which allowed them to sidestep all of the inherent problems with trying to share GPU memory, but too much to hope for. > No. It's gone on Gen11 and later. :/ And no replacement yet. I've been relatively distanced from the nitty-gritty parts of GPU virt for a couple of years, but how/why did the end up pushing for root SR-IOV support for Xe along with patches for ReBAR at the same time as they dropped this? https://www.intel.com/content/www/us/en/support/articles/000058558/graphics.html https://www.intel.com/content/www/us/en/support/articles/000... I guess they'll gate it behind enterprise dGPU SKUs when/if they ever release. Welp. > The OSS NV KM stack doesn't support GPU virt at all yet. I would have thought this also. As bad as the doc in the link is, it does explicitly reference OpenRM verification. I wouldn't trust nVidia's "big ball of hardly-commented source" for anything yet, but at least it is the "real" driver and not Nouveau, so I'd expect GPU virt to be supported, even if the code may be inscrutable. I'm really just waiting for the HN link to confirm/deny the long-held suspicion that nVidia suddenly got really good around the same time SGI was in its death throes, and that the reason they resisted opening their drivers was due to routines with... questionable IP.
- ArcVRArthur 4y ago
  Intel's Xe embedded SKUs are supported: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support We did some things to enable GPU virtualization on older driver revisions for most Nvidia consumer GPUs and some AMD GPUs as well. Here's that post if you'd like to take a look: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 We're building a free/libre (AGPLv3 & GPLv2) virtualization stack intended to support i915 (Intel) and Nvidia (OpenRM). We're planning support for Tenstorrent TPUs and in the longer term possibly AMD GPUs. Here's a full summary of how GPU virtualization actually works under various modes from our wiki. This includes VFIO-Mediated Devices (Mdev), Single Root I/O Virtualization (SR-IOV), and Intel's new Scalable I/O Virtualization (SIOV): https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals
  - yolo4553 4y ago
    Curious about Tenstorrent. Did you ever get your hands on any of their hardware? We are just getting excuses from them since several months and have not yet received even a single unit for evaluation.
    - ArcVRArthur 4y ago
      I believe we're expecting units in a few weeks.
  - evol262 4y ago
    > Intel's Xe embedded SKUs are supported: > https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support I'm not trying to beat a dead horse here, but your matrix links to an archive of Intel's community forum, where it's a basic question, about Windows. https://archive.ph/0McAE#selection-4883.0-4943.35 https://archive.ph/0McAE#selection-4883.0-4943.35 Then a link to Intel's LTS Linux repo. Do your scripts, if it's Xe, actually clone/build this and boot into it? What does "supported" mean in this sense? Do you replace the user's kernel with the intel-lts fork? If so, do you tell them? > We did some things to enable GPU virtualization on older driver revisions for most Nvidia consumer GPUs and some AMD GPUs as well. Here's that post if you'd like to take a look: You have some patches to drivers. Which is great. But "we did some things" on HN is best followed with "here's what we did from a technical level" or "here's the source", not "here's how to install our product". https://github.com/Arc-Compute/LibVF.IO/tree/master/patches https://github.com/Arc-Compute/LibVF.IO/tree/master/patches Is there anything at all here which isn't applicable to other KVM-based virt platforms? It looks like not. > We're building a free/libre (AGPLv3 & GPLv2) virtualization stack intended to support i915 (Intel) and Nvidia (OpenRM). Again, this is great. But please be clear that "we're building a free/libre virtualization stack" is "we are building a very opinionated wrapper around qemu+KVM which is not using libvirt bindings for some reason". The "stack" definitely looks like "some utility scripts to make this easier". Is there a project homepage with a roadmap, goals, and issues/bugs, and so on?
ArcVRArthur 4y ago
Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I am happy to say though that Intel now uses SR-IOV instead of GVT-g (deprecated since 9th generation Intel). Their replacement SR-IOV driver is now Open Source (recently made public): https://github.com/intel/linux-intel-lts/commit/41ef979f0894326c202473807a56b599a2f3d04e https://github.com/intel/linux-intel-lts/commit/41ef979f0894... Also to address the first comment in this thread - there are many inaccuracies here: Post-Ampere supports MIG and SR-IOV. VFIO-Mediated Devices (Mdev) are used both pre-Ampere and post-Ampere. This is how that works: https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals Also this tool has nothing to do with nvidia-cli or mdevctl. It defines available mdev devices in the mdevctl list, it does not replace mdevctl. For folks who are interested we also built LibVF.IO which enables vGPU/SR-IOV functionality on consumer GPUs: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 Here's a full list of supported GPUs from our wiki: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support
- evol262 4y ago
  > Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I am happy to say though that Intel now uses SR-IOV instead of GVT-g (deprecated since 9th generation Intel). Their replacement SR-IOV driver is now Open Source (recently made public): This would legitimately be ideal information to have in the readme/top-level link. > https://github.com/intel/linux-intel-lts/commit/41ef979f0894 https://github.com/intel/linux-intel-lts/commit/41ef979f0894 This is pretty unhelpful. Legitimately. It's not mainlined yet, there are zero userspace docs, etc. The patch looks like it will pretty much "just work" when/if Intel bothers to get it into mainline. Until then, a patched/forked kernel is needed. > Also to address the first comment in this thread - there are many inaccuracies here: > Post-Ampere supports MIG and SR-IOV. VFIO-Mediated Devices (Mdev) are used both pre-Ampere and post-Ampere. This is how that works: > https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals I maintained mdev support for a major KVM-based platform, but it's been a couple of years. That said, a link to how mdev internals work isn't useful to end-users, who just want to know "how do I partition my card"? As-in "which driver/utilities do I need to install"? > For folks who are interested we also built LibVF.IO which enables vGPU/SR-IOV functionality on consumer GPUs: > https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 > If you're interested in a full list of supported GPUs you can read the following page from our wiki: > https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support Is there some way in which LibVF.IO differs from just being a wrapper around KVM/qemu? Because the scripts do an awful lot of stuff to your host system, and arcd.nim appears to just call qemu anyway: https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libvfio/control/vm.nim https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libv... Sure, it also binds/removes mdev devices, which is a nice convenience, and you have a couple of patches applied to the nvidia driver sources, but asking users to blindly execute scripts they have to wade through to find out exactly what they're going to do in kernelspace, plus the system. It's... asking a lot. You're adding a virtual sound card, nim, shell aliases, samba, plasma, then blindly overwriting any kernel options the user has set without even the good grace of capturing them and appending them: https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/funcs-libvfio.sh#L184 https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/... > Also finally this tool has nothing to do with nvidia-cli or mdevctl. It defines available mdev devices in the mdevctl list, it does not replace mdevctl. It looks like it does a lot more than that, and less than that. It has nothing to do with nvidia-cli (which can also manage mdev devices), and "defines available mdev devices in the mdevctl list" only as a side effect of the fact that it's doing a whole bunch of other stuff to the system. I'm not trying to tear down your project, but honestly, the docs could be far, far better about what this actually does, how it does it, which pieces you actually need, etc. Because it certainly looks like all of the system configuration could be done with ansible/terraform instead of shell and published as an associated project and/or prerequisite steps where users would be informed of what's happening to their system. Similarly, it strongly appears that arcd could be more or less replaced by any given binding to the libvirt API, which would also allow the VMs to be easily migrated (assuming identical hardware), the libvirt XML shared with others, snapshotting, storage pooling, listed in virt-manager/virsh, and so on. As a basic open source citizen, this would also help in "giving credit where credit is due". Bravo for stitching all of this together, but the project pages/repo certainly make it seem as if LibVF.IO/GVM did the work, and that's not really honest. GVM, rather than "a GPU Virtual Machine ..." is Haskell bindings for the RMAPI. LibVF.IO is "Utilities KVM+qemu VMs with vGPU passthrough for humans" If there's a mistake here, please correct it, but going through the repos, it looks like a whole lot of glue. There's nothing wrong with that. It's still valuable. It's just that all of the work you did in researching/testing/building it could (and arguably should) just as well go somewhere like where one of the original devs working on vfio/GPU passthrough posted a five page braindump of everything you'd ever need to know which people have been shamelessly cribbing from (on the Arch Wiki and others) for years, maybe without knowing: http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-1-hardware.html http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part... It would be extremely valuable to the community to document HOW end-users can tweak all of these knobs WITHOUT windmill slamming a bunch of packages from scripts in your repo and relying on your nim glue to do it.

by my123 4y ago

evol262 4y ago

> More complex than that. MIG covers compute use cases only. For workloads where graphics are needed (or even the graphics APIs), you have to use preemptive scheduling, even on Ampere. Thanks for the correction! Shows how much I've really used it outside of "does nvidia-smi do the right thing, and can I map those to pods". I had assumed that, since nvidia-smi on Ampere did both GPU and CPU slicing, that maybe they found a solution which allowed them to sidestep all of the inherent problems with trying to share GPU memory, but too much to hope for. > No. It's gone on Gen11 and later. :/ And no replacement yet. I've been relatively distanced from the nitty-gritty parts of GPU virt for a couple of years, but how/why did the end up pushing for root SR-IOV support for Xe along with patches for ReBAR at the same time as they dropped this? https://www.intel.com/content/www/us/en/support/articles/000058558/graphics.html https://www.intel.com/content/www/us/en/support/articles/000... I guess they'll gate it behind enterprise dGPU SKUs when/if they ever release. Welp. > The OSS NV KM stack doesn't support GPU virt at all yet. I would have thought this also. As bad as the doc in the link is, it does explicitly reference OpenRM verification. I wouldn't trust nVidia's "big ball of hardly-commented source" for anything yet, but at least it is the "real" driver and not Nouveau, so I'd expect GPU virt to be supported, even if the code may be inscrutable. I'm really just waiting for the HN link to confirm/deny the long-held suspicion that nVidia suddenly got really good around the same time SGI was in its death throes, and that the reason they resisted opening their drivers was due to routines with... questionable IP.

ArcVRArthur 4y ago
Intel's Xe embedded SKUs are supported: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support We did some things to enable GPU virtualization on older driver revisions for most Nvidia consumer GPUs and some AMD GPUs as well. Here's that post if you'd like to take a look: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 We're building a free/libre (AGPLv3 & GPLv2) virtualization stack intended to support i915 (Intel) and Nvidia (OpenRM). We're planning support for Tenstorrent TPUs and in the longer term possibly AMD GPUs. Here's a full summary of how GPU virtualization actually works under various modes from our wiki. This includes VFIO-Mediated Devices (Mdev), Single Root I/O Virtualization (SR-IOV), and Intel's new Scalable I/O Virtualization (SIOV): https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals
- yolo4553 4y ago
  Curious about Tenstorrent. Did you ever get your hands on any of their hardware? We are just getting excuses from them since several months and have not yet received even a single unit for evaluation.
  - ArcVRArthur 4y ago
    I believe we're expecting units in a few weeks.
- evol262 4y ago
  > Intel's Xe embedded SKUs are supported: > https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support I'm not trying to beat a dead horse here, but your matrix links to an archive of Intel's community forum, where it's a basic question, about Windows. https://archive.ph/0McAE#selection-4883.0-4943.35 https://archive.ph/0McAE#selection-4883.0-4943.35 Then a link to Intel's LTS Linux repo. Do your scripts, if it's Xe, actually clone/build this and boot into it? What does "supported" mean in this sense? Do you replace the user's kernel with the intel-lts fork? If so, do you tell them? > We did some things to enable GPU virtualization on older driver revisions for most Nvidia consumer GPUs and some AMD GPUs as well. Here's that post if you'd like to take a look: You have some patches to drivers. Which is great. But "we did some things" on HN is best followed with "here's what we did from a technical level" or "here's the source", not "here's how to install our product". https://github.com/Arc-Compute/LibVF.IO/tree/master/patches https://github.com/Arc-Compute/LibVF.IO/tree/master/patches Is there anything at all here which isn't applicable to other KVM-based virt platforms? It looks like not. > We're building a free/libre (AGPLv3 & GPLv2) virtualization stack intended to support i915 (Intel) and Nvidia (OpenRM). Again, this is great. But please be clear that "we're building a free/libre virtualization stack" is "we are building a very opinionated wrapper around qemu+KVM which is not using libvirt bindings for some reason". The "stack" definitely looks like "some utility scripts to make this easier". Is there a project homepage with a roadmap, goals, and issues/bugs, and so on?

ArcVRArthur 4y ago

Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I am happy to say though that Intel now uses SR-IOV instead of GVT-g (deprecated since 9th generation Intel). Their replacement SR-IOV driver is now Open Source (recently made public): https://github.com/intel/linux-intel-lts/commit/41ef979f0894326c202473807a56b599a2f3d04e https://github.com/intel/linux-intel-lts/commit/41ef979f0894... Also to address the first comment in this thread - there are many inaccuracies here: Post-Ampere supports MIG and SR-IOV. VFIO-Mediated Devices (Mdev) are used both pre-Ampere and post-Ampere. This is how that works: https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals Also this tool has nothing to do with nvidia-cli or mdevctl. It defines available mdev devices in the mdevctl list, it does not replace mdevctl. For folks who are interested we also built LibVF.IO which enables vGPU/SR-IOV functionality on consumer GPUs: https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 Here's a full list of supported GPUs from our wiki: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support

evol262 4y ago
> Hey, OP here. Sorry I didn't get to this sooner as I posted this just before falling asleep. This comment basically hits the nail on the head in most areas. I am happy to say though that Intel now uses SR-IOV instead of GVT-g (deprecated since 9th generation Intel). Their replacement SR-IOV driver is now Open Source (recently made public): This would legitimately be ideal information to have in the readme/top-level link. > https://github.com/intel/linux-intel-lts/commit/41ef979f0894 https://github.com/intel/linux-intel-lts/commit/41ef979f0894 This is pretty unhelpful. Legitimately. It's not mainlined yet, there are zero userspace docs, etc. The patch looks like it will pretty much "just work" when/if Intel bothers to get it into mainline. Until then, a patched/forked kernel is needed. > Also to address the first comment in this thread - there are many inaccuracies here: > Post-Ampere supports MIG and SR-IOV. VFIO-Mediated Devices (Mdev) are used both pre-Ampere and post-Ampere. This is how that works: > https://openmdev.io/index.php/Mediated_Device_Internals https://openmdev.io/index.php/Mediated_Device_Internals I maintained mdev support for a major KVM-based platform, but it's been a couple of years. That said, a link to how mdev internals work isn't useful to end-users, who just want to know "how do I partition my card"? As-in "which driver/utilities do I need to install"? > For folks who are interested we also built LibVF.IO which enables vGPU/SR-IOV functionality on consumer GPUs: > https://news.ycombinator.com/item?id=28944426 https://news.ycombinator.com/item?id=28944426 > If you're interested in a full list of supported GPUs you can read the following page from our wiki: > https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support Is there some way in which LibVF.IO differs from just being a wrapper around KVM/qemu? Because the scripts do an awful lot of stuff to your host system, and arcd.nim appears to just call qemu anyway: https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libvfio/control/vm.nim https://github.com/Arc-Compute/LibVF.IO/blob/master/src/libv... Sure, it also binds/removes mdev devices, which is a nice convenience, and you have a couple of patches applied to the nvidia driver sources, but asking users to blindly execute scripts they have to wade through to find out exactly what they're going to do in kernelspace, plus the system. It's... asking a lot. You're adding a virtual sound card, nim, shell aliases, samba, plasma, then blindly overwriting any kernel options the user has set without even the good grace of capturing them and appending them: https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/funcs-libvfio.sh#L184 https://github.com/Arc-Compute/LibVF.IO/blob/master/scripts/... > Also finally this tool has nothing to do with nvidia-cli or mdevctl. It defines available mdev devices in the mdevctl list, it does not replace mdevctl. It looks like it does a lot more than that, and less than that. It has nothing to do with nvidia-cli (which can also manage mdev devices), and "defines available mdev devices in the mdevctl list" only as a side effect of the fact that it's doing a whole bunch of other stuff to the system. I'm not trying to tear down your project, but honestly, the docs could be far, far better about what this actually does, how it does it, which pieces you actually need, etc. Because it certainly looks like all of the system configuration could be done with ansible/terraform instead of shell and published as an associated project and/or prerequisite steps where users would be informed of what's happening to their system. Similarly, it strongly appears that arcd could be more or less replaced by any given binding to the libvirt API, which would also allow the VMs to be easily migrated (assuming identical hardware), the libvirt XML shared with others, snapshotting, storage pooling, listed in virt-manager/virsh, and so on. As a basic open source citizen, this would also help in "giving credit where credit is due". Bravo for stitching all of this together, but the project pages/repo certainly make it seem as if LibVF.IO/GVM did the work, and that's not really honest. GVM, rather than "a GPU Virtual Machine ..." is Haskell bindings for the RMAPI. LibVF.IO is "Utilities KVM+qemu VMs with vGPU passthrough for humans" If there's a mistake here, please correct it, but going through the repos, it looks like a whole lot of glue. There's nothing wrong with that. It's still valuable. It's just that all of the work you did in researching/testing/building it could (and arguably should) just as well go somewhere like where one of the original devs working on vfio/GPU passthrough posted a five page braindump of everything you'd ever need to know which people have been shamelessly cribbing from (on the Arch Wiki and others) for years, maybe without knowing: http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-1-hardware.html http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part... It would be extremely valuable to the community to document HOW end-users can tweak all of these knobs WITHOUT windmill slamming a bunch of packages from scripts in your repo and relying on your nim glue to do it.