6 ms·

Sharing a one/multiple physical GPUs with multiple virtual environments?

st_goliath 3y ago
I think you mean PCI pass-through? That allows you to isolate PCI resources and pass them through directly to the guest VM. The host doesn't need to have a driver or understand what that device is at all. Ideally the host should not even touch the device during boot, e.g. if it is a graphics cards. Virtio GPU is something different, namely a para-virtualized graphics adapter that uses the VirtIO guest-to-host communication to expose hardware accelerated graphics to the VM. For instance, on the guest side, you could have Mesa3D implementing OpenGL userspace and internally using its "Virgl" driver (or "Venus", for Vulkan) that serializes a command stream, sends that through a guest side kernel driver that passes it on through the VirtIO device to Qemu on the host side, that then uses existing OpenGL userspace graphics APIs to do the actual accelerated rendering.
- actionfromafar 3y ago
  Can this actually be done today, or would a lot things have to be implemented in that chain?
  - rwmj 3y ago
    Arch's wiki (as usual) is the best summary of the situation: https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceleration https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceler...
  - st_goliath 3y ago
    According to the Arch Wiki, the Linux side is "fairly mature"[0]. The Mesa3D documentation leaves a similar impression regarding OpenGL[1], mentioning that the Linux drivers, including the ones for 3D support, are mainline since 4.4 (released in 2016[2]) with Qemu support since version 2.5 (released around 2015[2]), so I guess, the assessment from the Arch Wiki sounds about right. The Venus driver for Vulkan is described as "experimental" [4], with Linux kernel side support merged in version 5.16 (release in early 2022[5]). RedHat is developing VirtIO drivers for Windows[6], but apparently they gave up the work on the OpenGL ICD driver in favor of Vulkan[7]? That said, I have no clue what the situation on various BSDs looks like. I guess on OpenBSD, this is a first step towards making this chain work? [0] https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceleration#Virgil3d_virtio-gpu_paravirtualized_device_driver https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceler... [1] https://docs.mesa3d.org/drivers/virgl.html https://docs.mesa3d.org/drivers/virgl.html [2] https://kernelnewbies.org/Linux_4.4 https://kernelnewbies.org/Linux_4.4 [3] https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg02731.html https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg027... [4] https://docs.mesa3d.org/drivers/venus.html https://docs.mesa3d.org/drivers/venus.html [5] https://kernelnewbies.org/Linux_5.16 https://kernelnewbies.org/Linux_5.16 [6] https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_... [7] https://lists.freedesktop.org/archives/virglrenderer-devel/2021-January/001897.html https://lists.freedesktop.org/archives/virglrenderer-devel/2...
- FloatArtifact 3y ago
  I believe with pass through it dedicates the entire physical device to the to a single virtual environment. I was hoping there's progress here to share across multiple virtual environments with even a single physical gpu.
  - loeg 3y ago
    Pass-through dedicates an entire d/b/s/f to a guest OS, but you could plausibly have a single GPU "device" (slot) providing multiple "functions," with the hardware multiplexed between guest consumers internally ("Multi-Function Devices"). I believe some SmartNICs do this? "SR-IOV" is a related concept.
crest 3y ago
This looks like a para-virtualised GPU driver and probably more similar to accelerated indirect OpenGL rendering than PCI(e) passthrough.

I think you mean PCI pass-through? That allows you to isolate PCI resources and pass them through directly to the guest VM. The host doesn't need to have a driver or understand what that device is at all. Ideally the host should not even touch the device during boot, e.g. if it is a graphics cards. Virtio GPU is something different, namely a para-virtualized graphics adapter that uses the VirtIO guest-to-host communication to expose hardware accelerated graphics to the VM. For instance, on the guest side, you could have Mesa3D implementing OpenGL userspace and internally using its "Virgl" driver (or "Venus", for Vulkan) that serializes a command stream, sends that through a guest side kernel driver that passes it on through the VirtIO device to Qemu on the host side, that then uses existing OpenGL userspace graphics APIs to do the actual accelerated rendering.

actionfromafar 3y ago
Can this actually be done today, or would a lot things have to be implemented in that chain?
- rwmj 3y ago
  Arch's wiki (as usual) is the best summary of the situation: https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceleration https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceler...
- st_goliath 3y ago
  According to the Arch Wiki, the Linux side is "fairly mature"[0]. The Mesa3D documentation leaves a similar impression regarding OpenGL[1], mentioning that the Linux drivers, including the ones for 3D support, are mainline since 4.4 (released in 2016[2]) with Qemu support since version 2.5 (released around 2015[2]), so I guess, the assessment from the Arch Wiki sounds about right. The Venus driver for Vulkan is described as "experimental" [4], with Linux kernel side support merged in version 5.16 (release in early 2022[5]). RedHat is developing VirtIO drivers for Windows[6], but apparently they gave up the work on the OpenGL ICD driver in favor of Vulkan[7]? That said, I have no clue what the situation on various BSDs looks like. I guess on OpenBSD, this is a first step towards making this chain work? [0] https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceleration#Virgil3d_virtio-gpu_paravirtualized_device_driver https://wiki.archlinux.org/title/QEMU/Guest_graphics_acceler... [1] https://docs.mesa3d.org/drivers/virgl.html https://docs.mesa3d.org/drivers/virgl.html [2] https://kernelnewbies.org/Linux_4.4 https://kernelnewbies.org/Linux_4.4 [3] https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg02731.html https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg027... [4] https://docs.mesa3d.org/drivers/venus.html https://docs.mesa3d.org/drivers/venus.html [5] https://kernelnewbies.org/Linux_5.16 https://kernelnewbies.org/Linux_5.16 [6] https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_... [7] https://lists.freedesktop.org/archives/virglrenderer-devel/2021-January/001897.html https://lists.freedesktop.org/archives/virglrenderer-devel/2...
FloatArtifact 3y ago
I believe with pass through it dedicates the entire physical device to the to a single virtual environment. I was hoping there's progress here to share across multiple virtual environments with even a single physical gpu.
- loeg 3y ago
  Pass-through dedicates an entire d/b/s/f to a guest OS, but you could plausibly have a single GPU "device" (slot) providing multiple "functions," with the hardware multiplexed between guest consumers internally ("Multi-Function Devices"). I believe some SmartNICs do this? "SR-IOV" is a related concept.

crest 3y ago

This looks like a para-virtualised GPU driver and probably more similar to accelerated indirect OpenGL rendering than PCI(e) passthrough.