5 ms·

Finally! Every time I need to run a virtual machine, I choose libvirt because it's more performant and easy to deal with than Virtualbox (no kernel module, etc

by sph 3y ago

Finally! Every time I need to run a virtual machine, I choose libvirt because it's more performant and easy to deal with than Virtualbox (no kernel module, etc.), but the GUI choices are pretty terrible. The "best" libvirt GUI is virt-manager and it's very, very buggy and lacking features (i.e. doesn't play nice with HiDPI screens, no way of configuring IPv6, etc.) Many times I have caved and chosen VirtualBox simply because at least it feels nice to use, even if not as performant as libvirt/kvm. Not anymore!

blitzclone 3y ago
Great! What guests do you typically run where you see better performance with libvirt/kvm?
- sph 3y ago
  Mostly Linux, but also Windows when I had a VFIO passthrough setup. I don't think it's even possible to set it up with Virtualbox to have decent enough performance.
  - blitzclone 3y ago
    You can also setup VFIO in VirtualBox/KVM. We haven't polished it yet though. You can check the video here to see GPU virtualization in action: https://www.cyberus-technology.de/products/hypervisor https://www.cyberus-technology.de/products/hypervisor (Don't mind the English, we are not native speakers. :)
    - madushan1000 3y ago
      I'm really curious about this, is it gpu para-virtualization or actual VFIO requiring built in support from the gpu hardware?
      - blitzclone 3y ago
        We have used this with recent Intel GPUs that support SR-IOV. This is what you see in the video on the cyberus website. Intel hasn't managed to upstream the drivers for this yet and you have to piece together things, which is very unpleasant. But we are there to help, if someone wants to use this in a professional setting. We used to have support for Intel GVT-g GPU virtualization as well, which was more of a software solution. This doesn't work with modern Intel GPUs anymore.
        madushan1000 3y ago
        Thank you for the info. I use AMD consumer GPUs, none of them unfortunately support SR-IOV afaik, there are some developments from google around virtio-gpu(DRM native context) I've been following, I was hoping this was something similar.
        tyfon 3y ago
        I wonder if it would be possible to use DXVK here for windows guests to play those unruly games. That is have a pass through direct x driver in windows that sends all the commands to dxvk which either sends the image back to the vm or renders directly on the screen.
eVeechu7 3y ago
I thought virt manager was ok but honestly your complaints about it are specific and fair.
- eek2121 3y ago
  Virtual box has graphical configuration for a ton of different options. It also “just works” in many cases and is relatively easy to use. I am surprised the open source community has not built better gui tools, and no project, closed or open has made configuring pcie passthrough easy. I have always wanted to be able to run Windows in a virtualized session with my GPU for gaming, and use my onboard APU for the Linux host, but the configuration is daunting, and many of the games I play today don’t work on linux thanks to anticheat or DRM.
  - photonbeam 3y ago
    I wish there was a port of UTM to linux
    - starkparker 3y ago
      Gnome Boxes is an attempt at a similar interface, but yeah, it's not quite as polished.
      - westurner 3y ago
        virt-manager supports more complex libvirt XML configurations, can also manage VMs created by Gnome Boxes, but doesn't yet have IOMMU/PCIE passthrough with OVMF UEFI device selector and vm configuration gui: https://virt-manager.org/ https://virt-manager.org/
  - AnthonyMouse 3y ago
    > I am surprised the open source community has not built better gui tools This is the thing that isn't surprising. In order to make a better GUI tool, you have to understand how the internals work. Then you don't care to use a GUI tool because you know how to use the command line or edit the configuration files. The people who want better GUI tools are the people who don't know how to make them. In order to change this, the people who want GUI tools but don't know how to make them have to provide some incentive (typically money) to the people who know how to make them but don't want them, e.g. via donations or some kind of commercial agreement. And if they stand around and wonder why nobody else has fixed their problem for them instead of doing the thing that causes it to be fixed, an object at rest tends to remain at rest.
WhyNotHugo 3y ago
virt-manager plays fine with hidpi on Wayland. On the opposite side, VirtualBox’s GUI is super buggy in Wayland. It’s basically the opposite for both. I use virt-manager because the GUI is simpler (and setting up virtualbox is a nightmare anyway). Regardless, this feature is a step in the right direction. I’m wondering if distributions will pick it up or if it will ever be integrated upstream.
- _cenw 3y ago
  That's a very recent change to virt-manager, so recent it's on nixos-unstable, but not on nixos-23.11. So it might be in Fedora 39 and some rolling-release distros... and nowhere else.
  - melvyn2 3y ago
    Could you link the change? At least a few days ago virt-manager still seemed to have scaling issues with guest displays, on nixos-unstable. I had viewer scaling on though as a workaround, so maybe I just didn’t notice.
    - _cenw 3y ago
      It's not perfect. On older versions, it'd sometimes add a black border the size of your scaling factor. It still reports your window size times your scaling factor as internal resolution with guest additions though.
_chu1 3y ago
100% agree about the terrible GUI choices. That actually turned me away completely from libvirt for virtual machines because when I actually needed to work with virtual machines I didn't want to finick around with CLIs and I stuck to VMware until recently. (at the time I ran Windows 10 on my main PC and Windows has a pretty terrible CLI)
pkulak 3y ago
Have you tried Gnome Boxes?
- eek2121 3y ago
  I have, it is very limited compared to Virtual Box.
iam-TJ 3y ago
Regarding IPv6, there is support. In fact I run IPv6 only networks including for the hypervisors. $ virsh net-dumpxml default6 <network> <name>default6</name> <uuid>73590ea2-eb15-4e67-b104-319721bdf302</uuid> <forward mode='route'/> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:ff:a7:2d'/> <domain name='default6'/> <ip family='ipv6' address='2001:db8:ffff::1' prefix='48'> </ip> </network> One can also use DHCPv6 if required.
- remram 3y ago
  In virt-manager?
  - deusum 3y ago
    Virt-Manager does allow you to edit the xml config file by hand from within the interface. So, yes you can use ipv6; but no, it's not a convenient point and click gui interaction. I'd be curious to hear the specific reasoning behind it.
- sph 3y ago
  That's what I meant, I had to use virsh to set that up because it's not possible to do in the GUI.
  - iam-TJ 3y ago
    But it is! virt-manager v4.0.0 (Ubuntu 22.04 amd64) "QEMU/KVM" > "Details" (via double-click or context menu) > "Virtual Networks" tab > "+" (Add) button > "Create Virtual Network"/"Details" tab > Expand "IPv6 configuration" > check "Enable IPv6" > enter "Network" prefix and/or check "Enable DHCPv6" > enter "Start" and "End" addresses
rubatuga 3y ago
If you use a network bridge there's no reason to have to configure any IP addresses at all.
lathiat 3y ago
Yeah this has always been a bit of a pain. Since LXD recently launched a web interface that includes graphical console support (plus VM support some time before that, it's not just a container host anymore, it can do both), I have been using that for remote VMs now. But it still has quite a few rough edges, but nicer than libvirt at least.
k8svet 3y ago
virt-manager is the kind of software that frustrates me, has lost me VM (due to beraking SPICE beyond it already having been broken for years). Bug reports go ignored. Yet, when it works, it works great. And KVM is great. But Christ, it's a nightmare sometimes. Features lag behind libvirt, sometimes years. Just not great. The virt-viewer port to Windows never made it past POC+1, IMO, also. I'm honestly surprised there aren't more alternatives. Especially since SPICE has gstreamer instegration. Gstreamer can natively do webrtc video and data back channel. Someone could hack together a webrtc broweser-based VM viewer and I suspect there's a lot of interest, potentially money making things you could crate.

by sph 3y ago

blitzclone 3y ago

Great! What guests do you typically run where you see better performance with libvirt/kvm?

sph 3y ago
Mostly Linux, but also Windows when I had a VFIO passthrough setup. I don't think it's even possible to set it up with Virtualbox to have decent enough performance.
- blitzclone 3y ago
  You can also setup VFIO in VirtualBox/KVM. We haven't polished it yet though. You can check the video here to see GPU virtualization in action: https://www.cyberus-technology.de/products/hypervisor https://www.cyberus-technology.de/products/hypervisor (Don't mind the English, we are not native speakers. :)
  - madushan1000 3y ago
    I'm really curious about this, is it gpu para-virtualization or actual VFIO requiring built in support from the gpu hardware?
    - blitzclone 3y ago
      We have used this with recent Intel GPUs that support SR-IOV. This is what you see in the video on the cyberus website. Intel hasn't managed to upstream the drivers for this yet and you have to piece together things, which is very unpleasant. But we are there to help, if someone wants to use this in a professional setting. We used to have support for Intel GVT-g GPU virtualization as well, which was more of a software solution. This doesn't work with modern Intel GPUs anymore.
      - madushan1000 3y ago
        Thank you for the info. I use AMD consumer GPUs, none of them unfortunately support SR-IOV afaik, there are some developments from google around virtio-gpu(DRM native context) I've been following, I was hoping this was something similar.
      - tyfon 3y ago
        I wonder if it would be possible to use DXVK here for windows guests to play those unruly games. That is have a pass through direct x driver in windows that sends all the commands to dxvk which either sends the image back to the vm or renders directly on the screen.

eVeechu7 3y ago

I thought virt manager was ok but honestly your complaints about it are specific and fair.

eek2121 3y ago
Virtual box has graphical configuration for a ton of different options. It also “just works” in many cases and is relatively easy to use. I am surprised the open source community has not built better gui tools, and no project, closed or open has made configuring pcie passthrough easy. I have always wanted to be able to run Windows in a virtualized session with my GPU for gaming, and use my onboard APU for the Linux host, but the configuration is daunting, and many of the games I play today don’t work on linux thanks to anticheat or DRM.
- photonbeam 3y ago
  I wish there was a port of UTM to linux
  - starkparker 3y ago
    Gnome Boxes is an attempt at a similar interface, but yeah, it's not quite as polished.
    - westurner 3y ago
      virt-manager supports more complex libvirt XML configurations, can also manage VMs created by Gnome Boxes, but doesn't yet have IOMMU/PCIE passthrough with OVMF UEFI device selector and vm configuration gui: https://virt-manager.org/ https://virt-manager.org/
- AnthonyMouse 3y ago
  > I am surprised the open source community has not built better gui tools This is the thing that isn't surprising. In order to make a better GUI tool, you have to understand how the internals work. Then you don't care to use a GUI tool because you know how to use the command line or edit the configuration files. The people who want better GUI tools are the people who don't know how to make them. In order to change this, the people who want GUI tools but don't know how to make them have to provide some incentive (typically money) to the people who know how to make them but don't want them, e.g. via donations or some kind of commercial agreement. And if they stand around and wonder why nobody else has fixed their problem for them instead of doing the thing that causes it to be fixed, an object at rest tends to remain at rest.

WhyNotHugo 3y ago

virt-manager plays fine with hidpi on Wayland. On the opposite side, VirtualBox’s GUI is super buggy in Wayland. It’s basically the opposite for both. I use virt-manager because the GUI is simpler (and setting up virtualbox is a nightmare anyway). Regardless, this feature is a step in the right direction. I’m wondering if distributions will pick it up or if it will ever be integrated upstream.

_cenw 3y ago
That's a very recent change to virt-manager, so recent it's on nixos-unstable, but not on nixos-23.11. So it might be in Fedora 39 and some rolling-release distros... and nowhere else.
- melvyn2 3y ago
  Could you link the change? At least a few days ago virt-manager still seemed to have scaling issues with guest displays, on nixos-unstable. I had viewer scaling on though as a workaround, so maybe I just didn’t notice.
  - _cenw 3y ago
    It's not perfect. On older versions, it'd sometimes add a black border the size of your scaling factor. It still reports your window size times your scaling factor as internal resolution with guest additions though.

_chu1 3y ago

100% agree about the terrible GUI choices. That actually turned me away completely from libvirt for virtual machines because when I actually needed to work with virtual machines I didn't want to finick around with CLIs and I stuck to VMware until recently. (at the time I ran Windows 10 on my main PC and Windows has a pretty terrible CLI)

pkulak 3y ago

Have you tried Gnome Boxes?

eek2121 3y ago
I have, it is very limited compared to Virtual Box.

iam-TJ 3y ago

Regarding IPv6, there is support. In fact I run IPv6 only networks including for the hypervisors. $ virsh net-dumpxml default6 <network> <name>default6</name> <uuid>73590ea2-eb15-4e67-b104-319721bdf302</uuid> <forward mode='route'/> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:ff:a7:2d'/> <domain name='default6'/> <ip family='ipv6' address='2001:db8:ffff::1' prefix='48'> </ip> </network> One can also use DHCPv6 if required.

remram 3y ago
In virt-manager?
- deusum 3y ago
  Virt-Manager does allow you to edit the xml config file by hand from within the interface. So, yes you can use ipv6; but no, it's not a convenient point and click gui interaction. I'd be curious to hear the specific reasoning behind it.
sph 3y ago
That's what I meant, I had to use virsh to set that up because it's not possible to do in the GUI.
- iam-TJ 3y ago
  But it is! virt-manager v4.0.0 (Ubuntu 22.04 amd64) "QEMU/KVM" > "Details" (via double-click or context menu) > "Virtual Networks" tab > "+" (Add) button > "Create Virtual Network"/"Details" tab > Expand "IPv6 configuration" > check "Enable IPv6" > enter "Network" prefix and/or check "Enable DHCPv6" > enter "Start" and "End" addresses

rubatuga 3y ago

If you use a network bridge there's no reason to have to configure any IP addresses at all.

lathiat 3y ago

Yeah this has always been a bit of a pain. Since LXD recently launched a web interface that includes graphical console support (plus VM support some time before that, it's not just a container host anymore, it can do both), I have been using that for remote VMs now. But it still has quite a few rough edges, but nicer than libvirt at least.

k8svet 3y ago

virt-manager is the kind of software that frustrates me, has lost me VM (due to beraking SPICE beyond it already having been broken for years). Bug reports go ignored. Yet, when it works, it works great. And KVM is great. But Christ, it's a nightmare sometimes. Features lag behind libvirt, sometimes years. Just not great. The virt-viewer port to Windows never made it past POC+1, IMO, also. I'm honestly surprised there aren't more alternatives. Especially since SPICE has gstreamer instegration. Gstreamer can natively do webrtc video and data back channel. Someone could hack together a webrtc broweser-based VM viewer and I suspect there's a lot of interest, potentially money making things you could crate.