5 ms·

I'm looking for a hypervisor recommendation... I am hoping to virtualize at least two if not all three of my machines: Unraid (pass-thru HDDs); pfSense (pass-th

I'm looking for a hypervisor recommendation... I am hoping to virtualize at least two if not all three of my machines: Unraid (pass-thru HDDs); pfSense (pass-thru NIC); and Windows 10 (pass-thru Graphics). I am hoping to have Windows 10 running fast enough to do some light photo editing. Which hypervisor would you think to try first?

mastax 6y ago
Have you ruled out unraid? It has built-in VM management and supports pass-through.
- anonymouswacker 6y ago
  The only issue is I didn't want to lose my pfSense connection when Unraid reboots for upgrades.
  - sjwright 6y ago
    I have the same impulse to want to virtualise my router. I’ve repeatedly agonised over this but keep coming back to the decision that as cool as it would be, so long as internet uptime matters, the router probably needs to be separate hardware. You can get good, power efficient pfsense hardware for around $200 USD. A few bonus notes: —You could spend a little bit more and get hardware that's good enough to add a virtualisation layer to your dedicated router. —If your uplink is less than 300 megabit, and you have a managed switch with VLANs, your pfsense box doesn't need to be dual NIC. This could open the door for cheaper or re-purposed hardware.
    - koyote 6y ago
      I have had pfSense running on Proxmox for several months now. Upgraded to the latest pfSense version a couple of days ago and the total downtime was in the order of minutes. It's a bit too early to say whether it's a good idea or not; the main concern I have is that you have two points of failure instead of only one. Other than that worry it has been very smooth.
      - gonzo 6y ago
        The company that makes pfSense used to run its (pfSense) firewalls in ESXi, but then I got tired of having to wait for VMware to come up before the firewall(s) would come up. That, and CARP on VMware is ... tiresome.
smcleod 6y ago
I'd usually recommend just using KVM on Fedora/CentOS, but if you're not that Linux savy or don't want to spend time fiddling around I'd probably say give Proxmox a go, or XCP-ng if you want something that's easier to cluster between hosts but don't mind the storage not being quite as quick.
- op00to 6y ago
  You don’t have to be that Linux savvy to manage virtual machines with Cockpit.
lukevp 6y ago
I’m not that into hypervisors, but wouldn’t hyper-v be a good choice here? You could have near native windows graphics performance.
mike_d 6y ago
You really don't want to virtualize unRAID.
JoshTriplett 6y ago
KVM works well. You can use system-wide libvirt-based management to have the VMs automatically come up and go down with the host machine, and treat them like services. That said, there's value in having your router/firewall separate from other systems (virtual or otherwise), which will substantially reduce complexity and make it less likely that you accidentally make a system accessible on the wrong side.
- thaumaturgy 6y ago
  I've just recently spun up some VMs on my mid-range laptop with KVM. Once I got it up and running, I was pretty happy with the performance, but getting it there was not super easy. In particular, qemu and apparmor don't always play nicely together and the error messages you get by default don't tell you that it's an apparmor problem. Importing a VMDK package was also tricky, I ended up having to get familiar with some XML incantations to get the machine to import properly. Feedback from qemu or its friends was totally obtuse and unhelpful there too.
- aljarry 6y ago
  Do you know if it's possible in KVM to create separate virtual subnetwork managed by KVM (like: adding VMs to that network, reading their IPs) which is accessible from parent network? I got home network and wanted KVM to control which VM gets which IP, but I couldn't make them accessible from outside - without static IPs on VM side. When I use virtual bridge, KVM cannot set the IPs, but at least they are accessible from my home network.
  - rwmj 6y ago
    It's not a KVM-level thing unless you want to fiddle by hand with bridges and obscure command line options. libvirt can do it reasonably easily on top however (see "virsh net-list" etc).
- blinkingled 6y ago
  Cockpit with Virtual Machines plugin also works quite well for normal VM scenarios - create/edit/view console etc. works great.
tjoff 6y ago
I've done all of that on XenServer, or rather XCP-ng now. (pfsense (nic passthrough), freenas (SAS controller passthrough) and windows 10 with gpu passthrough) It doesn't have the same hype as the others but far more open. Biggest drawback is that the configuration software is windows-only (you can get it running in wine but it is a bit hacky, also xen-orchestra is a free (for personal use, if you build from source) web-configuration utility that one can use). Would not want to be at mercy of vmware. Also don't want to invest time into a system with such restrictions on use. Proxmox might be a decent alternative but I have a few systems and the cost would be prohibitive for hobby use.
- plam503711 6y ago
  XCP-ng Center (the Windows client) will disappear and Xen Orchestra is the only officially support client. In the next month, we'll bundle "XO Lite" into hosts to manage a simple machine without having to install anything!
  - tjoff 6y ago
    So there will be no free alternative left? Not sure if a gimped shareware-like(?) option sounds that appealing.
whatsmyusername 6y ago
Proxmox. ESXi client is windows only and most of KVM's clients are either linux only, cli only, or dogshit. Proxmox is great as long as you have the mental fortitude to ignore the tickler that pops up on login if you're on community support only.
- adamfeldman 6y ago
  https://github.com/foundObjects/pve-nag-buster https://github.com/foundObjects/pve-nag-buster
- ex3ndr 6y ago
  ESXi web is good. Why you need an app for this?
  - whatsmyusername 6y ago
    I dumped it years ago when my container work kicked off and they started paywalling everything behind vcenter. If web works with base ESXi great! If not then I'm not interested. I just don't work on a lot of companies in the position of needing VMWare anymore. Either they're all in on AWS or they're too small to justify the expense.
- pvtmert 6y ago
  Actually ESXi web works well for 'outside' managing VMs and for connecting 'inside' there is official app 'VMWare Remote Console' on AppStore I haven't actually used Proxmox but had X11/Xquartz on macOS, following enough to manage them. You don't even need to open X manually. It gets started when there is a connection. ssh -X user@host -- virt-manager & Also getting started on KVM on Debian is as easy as: apt install -y libvirt-bin virt-manager
- yakk0 6y ago
  the Windows ESXi client has been EOL for a while now and is not supported by current versions. It's all managed through the web or the esxcli commands now.

mastax 6y ago

Have you ruled out unraid? It has built-in VM management and supports pass-through.

anonymouswacker 6y ago
The only issue is I didn't want to lose my pfSense connection when Unraid reboots for upgrades.
- sjwright 6y ago
  I have the same impulse to want to virtualise my router. I’ve repeatedly agonised over this but keep coming back to the decision that as cool as it would be, so long as internet uptime matters, the router probably needs to be separate hardware. You can get good, power efficient pfsense hardware for around $200 USD. A few bonus notes: —You could spend a little bit more and get hardware that's good enough to add a virtualisation layer to your dedicated router. —If your uplink is less than 300 megabit, and you have a managed switch with VLANs, your pfsense box doesn't need to be dual NIC. This could open the door for cheaper or re-purposed hardware.
  - koyote 6y ago
    I have had pfSense running on Proxmox for several months now. Upgraded to the latest pfSense version a couple of days ago and the total downtime was in the order of minutes. It's a bit too early to say whether it's a good idea or not; the main concern I have is that you have two points of failure instead of only one. Other than that worry it has been very smooth.
    - gonzo 6y ago
      The company that makes pfSense used to run its (pfSense) firewalls in ESXi, but then I got tired of having to wait for VMware to come up before the firewall(s) would come up. That, and CARP on VMware is ... tiresome.

smcleod 6y ago

I'd usually recommend just using KVM on Fedora/CentOS, but if you're not that Linux savy or don't want to spend time fiddling around I'd probably say give Proxmox a go, or XCP-ng if you want something that's easier to cluster between hosts but don't mind the storage not being quite as quick.

op00to 6y ago
You don’t have to be that Linux savvy to manage virtual machines with Cockpit.

lukevp 6y ago

I’m not that into hypervisors, but wouldn’t hyper-v be a good choice here? You could have near native windows graphics performance.

mike_d 6y ago

You really don't want to virtualize unRAID.

JoshTriplett 6y ago

KVM works well. You can use system-wide libvirt-based management to have the VMs automatically come up and go down with the host machine, and treat them like services. That said, there's value in having your router/firewall separate from other systems (virtual or otherwise), which will substantially reduce complexity and make it less likely that you accidentally make a system accessible on the wrong side.

thaumaturgy 6y ago
I've just recently spun up some VMs on my mid-range laptop with KVM. Once I got it up and running, I was pretty happy with the performance, but getting it there was not super easy. In particular, qemu and apparmor don't always play nicely together and the error messages you get by default don't tell you that it's an apparmor problem. Importing a VMDK package was also tricky, I ended up having to get familiar with some XML incantations to get the machine to import properly. Feedback from qemu or its friends was totally obtuse and unhelpful there too.
aljarry 6y ago
Do you know if it's possible in KVM to create separate virtual subnetwork managed by KVM (like: adding VMs to that network, reading their IPs) which is accessible from parent network? I got home network and wanted KVM to control which VM gets which IP, but I couldn't make them accessible from outside - without static IPs on VM side. When I use virtual bridge, KVM cannot set the IPs, but at least they are accessible from my home network.
- rwmj 6y ago
  It's not a KVM-level thing unless you want to fiddle by hand with bridges and obscure command line options. libvirt can do it reasonably easily on top however (see "virsh net-list" etc).
blinkingled 6y ago
Cockpit with Virtual Machines plugin also works quite well for normal VM scenarios - create/edit/view console etc. works great.

tjoff 6y ago

I've done all of that on XenServer, or rather XCP-ng now. (pfsense (nic passthrough), freenas (SAS controller passthrough) and windows 10 with gpu passthrough) It doesn't have the same hype as the others but far more open. Biggest drawback is that the configuration software is windows-only (you can get it running in wine but it is a bit hacky, also xen-orchestra is a free (for personal use, if you build from source) web-configuration utility that one can use). Would not want to be at mercy of vmware. Also don't want to invest time into a system with such restrictions on use. Proxmox might be a decent alternative but I have a few systems and the cost would be prohibitive for hobby use.

plam503711 6y ago
XCP-ng Center (the Windows client) will disappear and Xen Orchestra is the only officially support client. In the next month, we'll bundle "XO Lite" into hosts to manage a simple machine without having to install anything!
- tjoff 6y ago
  So there will be no free alternative left? Not sure if a gimped shareware-like(?) option sounds that appealing.

whatsmyusername 6y ago

Proxmox. ESXi client is windows only and most of KVM's clients are either linux only, cli only, or dogshit. Proxmox is great as long as you have the mental fortitude to ignore the tickler that pops up on login if you're on community support only.

adamfeldman 6y ago
https://github.com/foundObjects/pve-nag-buster https://github.com/foundObjects/pve-nag-buster
ex3ndr 6y ago
ESXi web is good. Why you need an app for this?
- whatsmyusername 6y ago
  I dumped it years ago when my container work kicked off and they started paywalling everything behind vcenter. If web works with base ESXi great! If not then I'm not interested. I just don't work on a lot of companies in the position of needing VMWare anymore. Either they're all in on AWS or they're too small to justify the expense.
pvtmert 6y ago
Actually ESXi web works well for 'outside' managing VMs and for connecting 'inside' there is official app 'VMWare Remote Console' on AppStore I haven't actually used Proxmox but had X11/Xquartz on macOS, following enough to manage them. You don't even need to open X manually. It gets started when there is a connection. ssh -X user@host -- virt-manager & Also getting started on KVM on Debian is as easy as: apt install -y libvirt-bin virt-manager
yakk0 6y ago
the Windows ESXi client has been EOL for a while now and is not supported by current versions. It's all managed through the web or the esxcli commands now.