3 ms·

What are the practical use cases for this?

SXX 4y ago
Practical use would be rather niche. For instance easy splitting resources of single GPU between multiple VMs if you want to run isolated instances of ML software. Or if you're need to run automated tests for your game across different OS. Or tests for any software that use GPU acceleration. In reality most users of VFIO passthrough use it for gaming or creating multiseat systems. I guess it's will be true for vGPU feature too.
- ArcVRArthur 4y ago
  There are also some benefits in load balancing GPUs. For instance processes can be controlled with controlled groups (cgroups) and niceness/thread affinity to make sure scheduling / resource allocation is done equitably between processes on the system. Using GVM and 'app VMs' it is possible to do similar things with the GPU (restrict GPU processes to a user configurable slice of the GPU so it cannot deny service to other processes on the system which are also using the GPU): https://openmdev.io/index.php/Mdev-GPU#fbLen https://openmdev.io/index.php/Mdev-GPU#fbLen Since this type of load balancing was originally used in the datacenter where virtual machine multi-tenancy was the use case the Quality of Service (QoS) functions here are fairly robust.
ArcVRArthur 4y ago
For my use I tend to run a high performing Windows VM (very good with most games and other GPU accelerated programs - even chrome/adobe these days uses the GPU) inside Linux. This can also be used for various high assurance use cases such as OpenXT / QubesOS (security by compartmentalization). For example a laptop computer being used by an automotive company for CAD to keep their information safe from other programs on the system (like a browser in another VM) the prying eyes of competitors. I made a video of that here: https://news.ycombinator.com/item?id=32585333 https://news.ycombinator.com/item?id=32585333 We have also been working on something called "LIME Is Mediated Emulation" which is a Win64 binary compatibility layer similar to WINE but using vGPUs. You can read about that here: https://openmdev.io/index.php/LIME_Is_Mediated_Emulation https://openmdev.io/index.php/LIME_Is_Mediated_Emulation
- lvass 4y ago
  This is beautiful, you're doing the Lord's work. I absolutely loathe the current situation of MAC security being so complicated on desktops combined with graphical performance issues in compartmentalization. And surely a lot of folks are also stuck on a certain OS because GPU performance in VMs is terrible. This is very much needed.
  - ArcVRArthur 4y ago
    Thanks!! We'll do our best to keep improving things for everyone. Hopefully security by compartmentalization folks benefit from our work as well. I'll be going to QubesOS Summit so hopefully there will be more good conversations there. :)
digitallyfree 4y ago
While less for general consumer use (honestly most consumers don't use CPU virtualiztion either), this has strong potential in VDI applications. Today with CPU virtualization available everywhere one inexpensive machine can host many VMs all sharing common processing power. If GPU virtualization gains traction this could mean we could be doing the same with GPUs but in an open manner (currently vGPUs are only avaliable in expensive datacenter GPUs and the tech is proprietary). Personally I would like to go with full VDI at home if inexpensive vGPU tech was available - though I guess I'm not the average home user.
- Morgawr 4y ago
  > honestly most consumers don't use CPU virtualiztion either Not that they are aware of at least :)
  - ArcVRArthur 4y ago
    I was thinking this over in the past couple days and I think the words 'that they are aware' is really key here. Ideally if GPU virtualization were sufficiently widespread as is support today for Intel VT-d, and AMD-v (IOMMU APIs for hardware assisted CPU virtualization) then software could make use of these functions without the user being aware of it. We're in a situation similar to that of CPU virtualization without hardware assistance with the early Xenoservers project from Cambridge (what would later become the Xen hypervisor and XenSource company). At that time there was not widespread support for virtualization assistance on most CPUs, and as a result Xen used methods like ring de-privileging to place the entire guest in ring 3 (userspace and kernel) while the hypervisor ran in ring 0 in order to virtualize any ordinary CPU model - my understanding is these were known as PV-guests (paravirtual guests). Over time however CPU companies began to introduce widespread support for features like VT-d and AMD-v to all of their models of CPU which enabled VM-exits/context save-restore with the use of shadow registers rather than ring de-privileging while Intel added new 'virtualization enhancements' through feature suites like vPro (SGX2 for example) which were only available on certain models of CPU (for example Xeon devices). Xen would adopt VT-d and AMD-v as HVM-guests (Hardware assisted virtualization) as they became more common on ubiquitous hardware and at the same time commercial forks of Xen would take advantage of these vPro features (like SGX2) for enterprise and high security government use-cases: https://wiki.xenproject.org/wiki/Xen_Project_Software_Overview#Guest_Types https://wiki.xenproject.org/wiki/Xen_Project_Software_Overvi... Like before (around the time of the Xenoservers project) today we can effectively virtualize the GPU without hardware assistance mechanisms: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/Virtual_I/O_Internals#Mdev_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#Mdev_Mod... Since it's now practical to virtualize any GPU device (as was the case in the past with early Xen on CPUs supporting virtualization for various use-cases regardless of whether or not the hardware provided assistance mechanisms) it might then be time to start moving to a new paradigm of 'enterprise' vs. 'consumer' - in other words new 'virtualization enhancements' (similar to vPro on Intel's Xeons, ect..) are developed for enterprise GPUs (for example shadow page deduplication in VRAM, import/export of redundant objects between IO Virtual Address buffers, IOMMU protected balloon/deballoon, ect..) and basic hardware assistance mechanisms like SR-IOV & SIOV are enabled by default, across the board: https://openmdev.io/index.php/Virtual_I/O_Internals#SR-IOV_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#SR-IOV_M... https://openmdev.io/index.php/Virtual_I/O_Internals#SIOV_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#SIOV_Mod...

SXX 4y ago

Practical use would be rather niche. For instance easy splitting resources of single GPU between multiple VMs if you want to run isolated instances of ML software. Or if you're need to run automated tests for your game across different OS. Or tests for any software that use GPU acceleration. In reality most users of VFIO passthrough use it for gaming or creating multiseat systems. I guess it's will be true for vGPU feature too.

ArcVRArthur 4y ago
There are also some benefits in load balancing GPUs. For instance processes can be controlled with controlled groups (cgroups) and niceness/thread affinity to make sure scheduling / resource allocation is done equitably between processes on the system. Using GVM and 'app VMs' it is possible to do similar things with the GPU (restrict GPU processes to a user configurable slice of the GPU so it cannot deny service to other processes on the system which are also using the GPU): https://openmdev.io/index.php/Mdev-GPU#fbLen https://openmdev.io/index.php/Mdev-GPU#fbLen Since this type of load balancing was originally used in the datacenter where virtual machine multi-tenancy was the use case the Quality of Service (QoS) functions here are fairly robust.

ArcVRArthur 4y ago

For my use I tend to run a high performing Windows VM (very good with most games and other GPU accelerated programs - even chrome/adobe these days uses the GPU) inside Linux. This can also be used for various high assurance use cases such as OpenXT / QubesOS (security by compartmentalization). For example a laptop computer being used by an automotive company for CAD to keep their information safe from other programs on the system (like a browser in another VM) the prying eyes of competitors. I made a video of that here: https://news.ycombinator.com/item?id=32585333 https://news.ycombinator.com/item?id=32585333 We have also been working on something called "LIME Is Mediated Emulation" which is a Win64 binary compatibility layer similar to WINE but using vGPUs. You can read about that here: https://openmdev.io/index.php/LIME_Is_Mediated_Emulation https://openmdev.io/index.php/LIME_Is_Mediated_Emulation

lvass 4y ago
This is beautiful, you're doing the Lord's work. I absolutely loathe the current situation of MAC security being so complicated on desktops combined with graphical performance issues in compartmentalization. And surely a lot of folks are also stuck on a certain OS because GPU performance in VMs is terrible. This is very much needed.
- ArcVRArthur 4y ago
  Thanks!! We'll do our best to keep improving things for everyone. Hopefully security by compartmentalization folks benefit from our work as well. I'll be going to QubesOS Summit so hopefully there will be more good conversations there. :)

digitallyfree 4y ago

While less for general consumer use (honestly most consumers don't use CPU virtualiztion either), this has strong potential in VDI applications. Today with CPU virtualization available everywhere one inexpensive machine can host many VMs all sharing common processing power. If GPU virtualization gains traction this could mean we could be doing the same with GPUs but in an open manner (currently vGPUs are only avaliable in expensive datacenter GPUs and the tech is proprietary). Personally I would like to go with full VDI at home if inexpensive vGPU tech was available - though I guess I'm not the average home user.

Morgawr 4y ago
> honestly most consumers don't use CPU virtualiztion either Not that they are aware of at least :)
- ArcVRArthur 4y ago
  I was thinking this over in the past couple days and I think the words 'that they are aware' is really key here. Ideally if GPU virtualization were sufficiently widespread as is support today for Intel VT-d, and AMD-v (IOMMU APIs for hardware assisted CPU virtualization) then software could make use of these functions without the user being aware of it. We're in a situation similar to that of CPU virtualization without hardware assistance with the early Xenoservers project from Cambridge (what would later become the Xen hypervisor and XenSource company). At that time there was not widespread support for virtualization assistance on most CPUs, and as a result Xen used methods like ring de-privileging to place the entire guest in ring 3 (userspace and kernel) while the hypervisor ran in ring 0 in order to virtualize any ordinary CPU model - my understanding is these were known as PV-guests (paravirtual guests). Over time however CPU companies began to introduce widespread support for features like VT-d and AMD-v to all of their models of CPU which enabled VM-exits/context save-restore with the use of shadow registers rather than ring de-privileging while Intel added new 'virtualization enhancements' through feature suites like vPro (SGX2 for example) which were only available on certain models of CPU (for example Xeon devices). Xen would adopt VT-d and AMD-v as HVM-guests (Hardware assisted virtualization) as they became more common on ubiquitous hardware and at the same time commercial forks of Xen would take advantage of these vPro features (like SGX2) for enterprise and high security government use-cases: https://wiki.xenproject.org/wiki/Xen_Project_Software_Overview#Guest_Types https://wiki.xenproject.org/wiki/Xen_Project_Software_Overvi... Like before (around the time of the Xenoservers project) today we can effectively virtualize the GPU without hardware assistance mechanisms: https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/GPU_Support https://openmdev.io/index.php/Virtual_I/O_Internals#Mdev_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#Mdev_Mod... Since it's now practical to virtualize any GPU device (as was the case in the past with early Xen on CPUs supporting virtualization for various use-cases regardless of whether or not the hardware provided assistance mechanisms) it might then be time to start moving to a new paradigm of 'enterprise' vs. 'consumer' - in other words new 'virtualization enhancements' (similar to vPro on Intel's Xeons, ect..) are developed for enterprise GPUs (for example shadow page deduplication in VRAM, import/export of redundant objects between IO Virtual Address buffers, IOMMU protected balloon/deballoon, ect..) and basic hardware assistance mechanisms like SR-IOV & SIOV are enabled by default, across the board: https://openmdev.io/index.php/Virtual_I/O_Internals#SR-IOV_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#SR-IOV_M... https://openmdev.io/index.php/Virtual_I/O_Internals#SIOV_Mode https://openmdev.io/index.php/Virtual_I/O_Internals#SIOV_Mod...