12 ms·

I run a VFIO setup with a single GPU - the linux host is headless, and Windows runs on top with the GPU. Its pretty awesome. Windows runs at native performance

I run a VFIO setup with a single GPU - the linux host is headless, and Windows runs on top with the GPU. Its pretty awesome. Windows runs at native performance - no problem gaming or running other heavy workloads. The linux host acts as a devbox and runs a few other homeserver style services. It's difficult to set up right but it taught me a lot about VMs and hardware. Once you get it setup well enough, its relatively painless. Like I haven't messed with my VM settings in over a year, everything just continues to work smoothly. Including windows updates, driver upgrades, most online games with anti-cheat etc. If i upgrade my hardware, it might take a day or two of tinkering to get it back up. Based on my benchmarking it runs within ~5% of native perf. This is still the best guide IMO if you want to set it up - https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF. Single GPU is basically the same as dual GPU, except you have to ensure the linux host does not take over the GPU or load any drivers for it during boot.

cdata 5y ago
Which games do you play that require anti-cheat schemes? The last time I checked, many such games explicitly do not support running in VMs and you risk being banned by obfuscating that you are running the game in a guest OS.
- Google234 5y ago
  For EAC, they require a variety of signals to ban someone (unless it’s a obvious thing like detecting a known cheat.) An honest VM setup that doesn’t obfuscate probably counts as one potential signal that you are cheating, an obfuscated one that they are able detect might be a stronger signal.
- gnif 5y ago
  Most games work just fine, just a few odd ones that decided to block VMs don't such as Tarkov. Known to work without issue are titles such as * PUBG * Battlefield 1/3/4/5 * Titanfall 1&2 * Arma 3 * 7 Days to Die * Ark * Fortnite * Apex * Halo: The Master Chief Collection * Star Wars Squadrons And many many more.
  - gigatexal 5y ago
    The only games I play are Doom Eternal and Starcraft 2 — by chance do you know if those work or if there’s a list somewhere?
    - gnif 5y ago
      No idea about doom, but Starcraft 2 works for me fine.
    - himinlomax 5y ago
      SC2 works almost perfectly under Wine/Proton. Only issue is that a few custom maps crash, and there's a weird, minor performance issue at some point in the LOTV campaign menus (but not the game itself). Haven't had an issue in a ladder game in years. And I play SC2 a lot (too much).
      - gnif 5y ago
        "Almost" is the key word here, under a VFIO VM, it's 100% flawless.
    - zaptheimpaler 5y ago
      I recently finished the single player campaign on Doom Eternal - that worked perfect, though i havent tried multiplayer.
  - unholythree 5y ago
    Really, is that why Tarkov drops me when I try to start a match? At least CS:GO told me their anti-cheat hated my setup (Windows 10 in a Xenserver VM) and I was able to get a refund.
    - gnif 5y ago
      CS:GO works fine in a libvirt/KVM VM last I checked. As for Tarkov, see: https://mobile.twitter.com/TheBattlEye/status/1289027672186720263 https://mobile.twitter.com/TheBattlEye/status/12890276721867... Note that the game vendor selects what features they want to apply to their titles that are available. BattlEye allows you to stop people using VMs, if the game vendor opts in to this stupid feature.
- zaptheimpaler 5y ago
  Ive played PUBG, COD Warzone, Apex Legends, Quake Champions, maybe a few others. Of course it depends on the game but i think the perception of how many games don't work under VMs is a bit skewed. TBH every thread like this has the “but what about anti-cheat?” post which makes it seem like a big problem but i think only a tiny minority of games don’t work under VMs.
  - cdata 5y ago
    Thanks! That is more than I expected. FWIW I tried to install Valorant in a VM recently (because I'm tired of installing the root-kit anti-cheat on my main PC in order to play it). It simply fails to launch the anti-cheat, and so doesn't launch the game.
    - gnif 5y ago
      Yes, unfortunately Valorant is one that blocks VM usage... not that it has helped them any.
      - egeozcan 5y ago
        Anti-cheat measures end up blocking mostly the well-meaning players. I know it's sometimes too costly, but game studios should invest more on server-side security measures and mostly distrust the clients.
        Cloudef 5y ago
        The anti-cheat systems end up really just being more menance than use. Tons of money burned on something that cheaters will get around anyways.
        pbhjpbhj 5y ago
        It's one of those 'keeping honest people honest' things, of it were even easier I think even more people would do it?
Google234 5y ago
Better anti cheats won’t allow setups like this.
- lostlogin 5y ago
  ‘Better’? Why are VMs being blocked?
  - gnif 5y ago
    Because some cheats use a VM environment to hide from detection as they can act on the VM from outside of it. The solution though is not to ban VMs, but to push vendors like AMD and Intel to enable access to, and enforce usage of technologies like SEV if running inside a VM. https://www.kernel.org/doc/html/v5.6/virt/kvm/amd-memory-encryption.html https://www.kernel.org/doc/html/v5.6/virt/kvm/amd-memory-enc...
    - egeozcan 5y ago
      I wrote this another comment, but instead of getting too much into an arms race, studios should invest more on server-side anti-cheat mechanisms. You could correlate community feedback and some machine learning, while also picking easy-to-catch impossible actions.
      - atanasi 5y ago
        Server-side solutions don't catch all cheats. They can block actions that are impossible according to the game rules but they cannot prevent clients from disclosing too much information to the player about other players, or automating actions that are technically possible, like using aimbots.
        KronisLV 5y ago
        You can definitely handle some of those situations server side (the key word being "some") with enough engineering effort. In regards to player positions: check which player locations are occluded and wouldn't be visible through the geometry, then only send the valid ones for each player. Of course, doing this on high tick servers could prove to be computationally intensive. In regards to aimbots: the clients already send you information about where they're looking so that it can be displayed to other players. Attach some mouse movement metrics and from that you'll sometimes be able to infer the most naive aimbots instantly.
genewitch 5y ago
I do the vfio thing as well, do you happen to remember the registry editing you had to do to stop stuttering in games? There's a program that can set and unset all the stuff you need but evidently I deleted it. It's the only thing I am missing to build another one or rebuild the one I have. I wish I had believed it was going to work perfectly when I started and wrote everything down.
- gnif 5y ago
  Likely setting the NVidia drivers to use MSI (Message Signalled Interrupts)
- zaptheimpaler 5y ago
  I don’t recall doing any registry edits. Most of the perf work i remember doing was on the VM side - getting the little tweaks in libvirt settings, matching vCPU topology to physical, keeping VM cores from running linux system processes with GRUB flags, using a dedicated USB controller and sound card. i think i set the MSI stuff gnif mentioned as well though it wasn’t critical for me. Generally stuttering will be resolved by making sure time critical events (like interrupts) are delivered quickly to the guest. +1 on the writing stuff down :D I did it twice and documented pretty thoroughly the second time around, notes before each change, testing performance delta, notes about if it worked. It really helps.
  - deleted 5y ago
    [deleted]
- pizza234 5y ago
  I have a VFIO setup as well, and I'm not aware of any register setting correcting stuttering. There was an issue with AMD systems a few years ago, which caused microstuttering.
- panda-giddiness 5y ago
  I fixed my stuttering by enabling cpu tuning and setting the correct cpu topology [1]. [1] https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#CPU_pinning https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#CP...
ghostly_s 5y ago
The host is headless? So you're streaming the display via VNC or something to another system, or do you mean the host has a head but it's just being passed thru directly to the guest?
- gnif 5y ago
  The GPU is given to the guest, so the guest is driving the GPU directly making it a headless host.
  - coldtea 5y ago
    Does this also work for audio? E.g. could you run a high end DAW on the guest with the same performance? If not, is it something that might be added later?
- zaptheimpaler 5y ago
  The host PC has 1 graphics card, the host OS is running headless, and the windows OS is using the graphics card.
  - Aeolun 5y ago
    Does it boot the windows UI by default then? Or you just boot into a plain shell? Kind of defeats the point if your UI becomes windows again in my opinion.
- Cyph0n 5y ago
  Headless host means you can only access the host system via SSH or web interface (e.g., if you’re running Proxmox). The guest VM “owns” the single GPU.
wydfre 5y ago
You can't mention VFIO without mentioning the amazing subreddit, r/VFIO[0]. [0]: https://www.reddit.com/r/VFIO/ https://www.reddit.com/r/VFIO/
- aninteger 5y ago
  Even more amazing on the old interface: https://old.reddit.com/r/VFIO/ https://old.reddit.com/r/VFIO/
  - Crontab 5y ago
    Indeed. The new Reddit interest is horrid.
  - SkyMarshal 5y ago
    Or you can just go into your reddit settings and change back to the old interface permanently. I have that enabled and both these links are the same to me.
darkwater 5y ago
why don't you go the other way round, Windows native ane WSL2? Beside wanting to hack and learn things, obviouvsly.
- zaptheimpaler 5y ago
  Tried WSL, was buggy and had some issues with filesystem performance i think. I prefer having a 100% real linux & windows OSes that just work over slightly buggy workarounds. Plus i also host a lot of homeserver services (media server, SMB server, postgres db etc) on the linux host and its cleaner having those run on the host. If i want to do something very demanding on the host for example I can shutdown the VM.
  - hutrdvnj 5y ago
    Wouldn't it be easier to have 2 devices, one specifically designed for gaming and one running 24/7 for home server services. A cheap used Thinkpad could be used as linux device so it doesn't have to be expensive. Plus if you value your own time and effort you have to put in for your current setup, then it might actually be cheaper.
    - pizza234 5y ago
      It depends on what do you mean with "easier". Technically speaking, for machines where VFIO works, there is no maintenance, and the setup can be easy (excluding one significant issue due to a specific linux kernel upgrade, my VFIO setup procedure has been essentially the same for years, on multiple machines). So, assuming that VFIO setup setup/maintenance is very easy, having a second machine is just redundant. One convenience not to forget is safety. If a Windows VM gets infected for whatever reasons, rolling back the system is performed by literally deleting one file.
    - z3t4 5y ago
      You basically just need an extra gfx card (or even headless host). And why would you not want a fast Linux OS? Many SSD's and lots of RAM to make the IDE fast. Powerful CPU for compiling. Good GFX for better latency/hz.
porker 5y ago
This has been my dream setup for years, so that rather than a Windows host with Linux VMs I can run the other way round, enable ZFS and continually snapshot the Windows VM, and avoid the usual Windows failure as bits flip or sectors go bad.
Cyph0n 5y ago
Same setup here! I am running Proxmox on the host to streamline managing VMs and storage. Proxmox comes with a nice web GUI, which makes it very east to monitor system state. I have a Windows VM for gaming that owns the single Nvidia GPU. I also have a few Linux VMs for development (via VS Code remote) and media management. As far as storage goes, I don’t have anything too fancy. Proxmox is installed on an SSD. I have a second SSD for VM images. For all other storage (media, photos, VM image backups, etc.), I have a 3 disk ZFS pool consisting of a single RAID-Z1 vdev - yea, it’s risky, but losing the pool wouldn’t be the end of the world. One of the cool things about this kind of setup is being able to easily restore VMs from backup. Some time back, I accidentally screwed up my Windows install by enabling Hyper-V (nested virt). I panicked at first, but then remembered that I have daily snapshots of the VM. I had it back up and running within 10 minutes :) All in all, it was fun to setup and has been running very smoothly.
- zaptheimpaler 5y ago
  Yes I love that an entire VM is basically 2 files - one disk image, one libvirt config. Makes backup and restore so easy :)
- j1elo 5y ago
  Would you mind talking a bit about the small decisions you took for a Proxmox setup? I am slowly learning and planning my (very)small server setup. Things like: * Are the VM image backups you mentioned, done to your pool by means of the ZFS snapshots? Or done at the file-level with rsync or similar. * Do you make backups of the Proxmox installation? Similar as before: is Proxmox itself on a ZFS volume, so backups can be done just by doing ZFS snapshots? The installer lets me choose between an LVM-Thin + ext4, or a ZFS filesystem, and I was wondering whether to choose one or the other, for maximum convenience. * "Proxmox is installed on an SSD": isn't that a bit wasteful? I mean, doesn't the Proxmox system just take like 1 or 2 GB at most? I have a Lenovo ThinkCentre m910q which brings a 160GB M.2 NVMe disk, and another 320GB SSD disk... so I an in the process of deciding where to put each thing. Although for bigger storage I'm also considering if adding a 1 or 2 TB USB3 external disk would make sense (to store user backups like photos, documents, and also for the server's system backups)
mschuster91 5y ago
Which GPU do you have? I tried using a Mac Pro 5.1 as host, Debian Testing as OS and various Windows varieties as well as Linux (again Debian Testing) as guest - but I always, always run into that nasty PCI reset bug (https://www.nicksherlock.com/2020/11/working-around-the-amd-gpu-reset-bug-on-proxmox/ https://www.nicksherlock.com/2020/11/working-around-the-amd-...) :(
- MegaDeKay 5y ago
  Are you saying that that workaround doesn't help you? You could drop into their Discord on the vendor-reset channel and see if they can help you out. https://discord.gg/FJs9ufyu https://discord.gg/FJs9ufyu
isatty 5y ago
I use a vfio setup as well but with two GPUs (one integrated, one dedicated) and hotswap the dedicated GPU between the two whenever I’m playing games (I play on both Linux and Windows).
- keyb0ardninja 5y ago
  Is it possible to switch the GPU while the host is running, that is without rebooting?
  - isatty 5y ago
    It is, see: https://stty.io/2018/07/27/running-windows-with-pci-passthrough-on-gentoo-linux/ https://stty.io/2018/07/27/running-windows-with-pci-passthro... (Relevant scripts sub-heading)
    - keyb0ardninja 5y ago
      Thanks a lot, I'll check it out.
rbanffy 5y ago
If you have more than one GPU (even as in one on-board and one discrete) you can, with some effort, dedicate one of them to a VM and let the host use the other. Low-yield crypto mining is a common usage for that kind of arrangement, but I suppose VMs would work just fine too. Or go full Unix and attach a serial terminal to the console port and happy linuxing from there while you ignore the built-in gaming console running off a VM. This is more or less what every modern console does anyway - a hypervisor under everything. ;-)
chevill 5y ago
> most online games with anti-cheat etc. Any notable exceptions? What's the performance like compared to the same games in a native Windows install. I tried proton several months ago and I had to go back to Windows because it wasn't even close to good enough as far as compatability goes. I'd really love to abandon Windows as a desktop OS and just game in a VM though.
- Fabricio20 5y ago
  A pretty notable example is probably BattlEye, which is used by several major FPS games [1] like Tarkov, Rainbow Six and PUBG. They banned the use of virtual machines [2] back in 2020 and their AC solution does not work on Linux. Valve seems to be working on it [3], though their solution might be tailored to their SteamDeck hardware instead of a generic proton fix. [1]: https://www.battleye.com/ https://www.battleye.com/ [2]: https://twitter.com/TheBattlEye/status/1289027672186720263?s=20 https://twitter.com/TheBattlEye/status/1289027672186720263?s... [3]: https://partner.steamgames.com/doc/steamdeck/faq https://partner.steamgames.com/doc/steamdeck/faq
  - Nullabillity 5y ago
    PUBG uses EasyAntiCheat these days, rather than BattlEye.
- deleted 5y ago
  [deleted]
moondev 5y ago
I run the same setup but with esxi as the host OS. It's quite a bit easier to setup since esxi already runs headless by design, as well as easy PCI passthrough configuration via the host client html5 UI
MegaDeKay 5y ago
I have a VFIO setup as well but with dual GPUs. As you say, it is pretty awesome. I keep setting up other OS's for fun. I have an old nVidia NVS300 card that is supported natively under macOS, and also fired up a WindowsXP setup for grins. There were some interesting snags in getting each one going, and I learned a lot along the way. The Arch wiki is indeed spectacular.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
EchoReflection 5y ago
@zaptheimpaler thank you for sharing this.

cdata 5y ago

Which games do you play that require anti-cheat schemes? The last time I checked, many such games explicitly do not support running in VMs and you risk being banned by obfuscating that you are running the game in a guest OS.

Google234 5y ago
For EAC, they require a variety of signals to ban someone (unless it’s a obvious thing like detecting a known cheat.) An honest VM setup that doesn’t obfuscate probably counts as one potential signal that you are cheating, an obfuscated one that they are able detect might be a stronger signal.
gnif 5y ago
Most games work just fine, just a few odd ones that decided to block VMs don't such as Tarkov. Known to work without issue are titles such as * PUBG * Battlefield 1/3/4/5 * Titanfall 1&2 * Arma 3 * 7 Days to Die * Ark * Fortnite * Apex * Halo: The Master Chief Collection * Star Wars Squadrons And many many more.
- gigatexal 5y ago
  The only games I play are Doom Eternal and Starcraft 2 — by chance do you know if those work or if there’s a list somewhere?
  - gnif 5y ago
    No idea about doom, but Starcraft 2 works for me fine.
  - himinlomax 5y ago
    SC2 works almost perfectly under Wine/Proton. Only issue is that a few custom maps crash, and there's a weird, minor performance issue at some point in the LOTV campaign menus (but not the game itself). Haven't had an issue in a ladder game in years. And I play SC2 a lot (too much).
    - gnif 5y ago
      "Almost" is the key word here, under a VFIO VM, it's 100% flawless.
  - zaptheimpaler 5y ago
    I recently finished the single player campaign on Doom Eternal - that worked perfect, though i havent tried multiplayer.
- unholythree 5y ago
  Really, is that why Tarkov drops me when I try to start a match? At least CS:GO told me their anti-cheat hated my setup (Windows 10 in a Xenserver VM) and I was able to get a refund.
  - gnif 5y ago
    CS:GO works fine in a libvirt/KVM VM last I checked. As for Tarkov, see: https://mobile.twitter.com/TheBattlEye/status/1289027672186720263 https://mobile.twitter.com/TheBattlEye/status/12890276721867... Note that the game vendor selects what features they want to apply to their titles that are available. BattlEye allows you to stop people using VMs, if the game vendor opts in to this stupid feature.
zaptheimpaler 5y ago
Ive played PUBG, COD Warzone, Apex Legends, Quake Champions, maybe a few others. Of course it depends on the game but i think the perception of how many games don't work under VMs is a bit skewed. TBH every thread like this has the “but what about anti-cheat?” post which makes it seem like a big problem but i think only a tiny minority of games don’t work under VMs.
- cdata 5y ago
  Thanks! That is more than I expected. FWIW I tried to install Valorant in a VM recently (because I'm tired of installing the root-kit anti-cheat on my main PC in order to play it). It simply fails to launch the anti-cheat, and so doesn't launch the game.
  - gnif 5y ago
    Yes, unfortunately Valorant is one that blocks VM usage... not that it has helped them any.
    - egeozcan 5y ago
      Anti-cheat measures end up blocking mostly the well-meaning players. I know it's sometimes too costly, but game studios should invest more on server-side security measures and mostly distrust the clients.
      - Cloudef 5y ago
        The anti-cheat systems end up really just being more menance than use. Tons of money burned on something that cheaters will get around anyways.
        pbhjpbhj 5y ago
        It's one of those 'keeping honest people honest' things, of it were even easier I think even more people would do it?

Google234 5y ago

Better anti cheats won’t allow setups like this.

lostlogin 5y ago
‘Better’? Why are VMs being blocked?
- gnif 5y ago
  Because some cheats use a VM environment to hide from detection as they can act on the VM from outside of it. The solution though is not to ban VMs, but to push vendors like AMD and Intel to enable access to, and enforce usage of technologies like SEV if running inside a VM. https://www.kernel.org/doc/html/v5.6/virt/kvm/amd-memory-encryption.html https://www.kernel.org/doc/html/v5.6/virt/kvm/amd-memory-enc...
  - egeozcan 5y ago
    I wrote this another comment, but instead of getting too much into an arms race, studios should invest more on server-side anti-cheat mechanisms. You could correlate community feedback and some machine learning, while also picking easy-to-catch impossible actions.
    - atanasi 5y ago
      Server-side solutions don't catch all cheats. They can block actions that are impossible according to the game rules but they cannot prevent clients from disclosing too much information to the player about other players, or automating actions that are technically possible, like using aimbots.
      - KronisLV 5y ago
        You can definitely handle some of those situations server side (the key word being "some") with enough engineering effort. In regards to player positions: check which player locations are occluded and wouldn't be visible through the geometry, then only send the valid ones for each player. Of course, doing this on high tick servers could prove to be computationally intensive. In regards to aimbots: the clients already send you information about where they're looking so that it can be displayed to other players. Attach some mouse movement metrics and from that you'll sometimes be able to infer the most naive aimbots instantly.

genewitch 5y ago

I do the vfio thing as well, do you happen to remember the registry editing you had to do to stop stuttering in games? There's a program that can set and unset all the stuff you need but evidently I deleted it. It's the only thing I am missing to build another one or rebuild the one I have. I wish I had believed it was going to work perfectly when I started and wrote everything down.

gnif 5y ago
Likely setting the NVidia drivers to use MSI (Message Signalled Interrupts)
zaptheimpaler 5y ago
I don’t recall doing any registry edits. Most of the perf work i remember doing was on the VM side - getting the little tweaks in libvirt settings, matching vCPU topology to physical, keeping VM cores from running linux system processes with GRUB flags, using a dedicated USB controller and sound card. i think i set the MSI stuff gnif mentioned as well though it wasn’t critical for me. Generally stuttering will be resolved by making sure time critical events (like interrupts) are delivered quickly to the guest. +1 on the writing stuff down :D I did it twice and documented pretty thoroughly the second time around, notes before each change, testing performance delta, notes about if it worked. It really helps.
- deleted 5y ago
  [deleted]
pizza234 5y ago
I have a VFIO setup as well, and I'm not aware of any register setting correcting stuttering. There was an issue with AMD systems a few years ago, which caused microstuttering.
panda-giddiness 5y ago
I fixed my stuttering by enabling cpu tuning and setting the correct cpu topology [1]. [1] https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#CPU_pinning https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#CP...

ghostly_s 5y ago

The host is headless? So you're streaming the display via VNC or something to another system, or do you mean the host has a head but it's just being passed thru directly to the guest?

gnif 5y ago
The GPU is given to the guest, so the guest is driving the GPU directly making it a headless host.
- coldtea 5y ago
  Does this also work for audio? E.g. could you run a high end DAW on the guest with the same performance? If not, is it something that might be added later?
zaptheimpaler 5y ago
The host PC has 1 graphics card, the host OS is running headless, and the windows OS is using the graphics card.
- Aeolun 5y ago
  Does it boot the windows UI by default then? Or you just boot into a plain shell? Kind of defeats the point if your UI becomes windows again in my opinion.
Cyph0n 5y ago
Headless host means you can only access the host system via SSH or web interface (e.g., if you’re running Proxmox). The guest VM “owns” the single GPU.

wydfre 5y ago

You can't mention VFIO without mentioning the amazing subreddit, r/VFIO[0]. [0]: https://www.reddit.com/r/VFIO/ https://www.reddit.com/r/VFIO/

aninteger 5y ago
Even more amazing on the old interface: https://old.reddit.com/r/VFIO/ https://old.reddit.com/r/VFIO/
- Crontab 5y ago
  Indeed. The new Reddit interest is horrid.
- SkyMarshal 5y ago
  Or you can just go into your reddit settings and change back to the old interface permanently. I have that enabled and both these links are the same to me.

darkwater 5y ago

why don't you go the other way round, Windows native ane WSL2? Beside wanting to hack and learn things, obviouvsly.

zaptheimpaler 5y ago
Tried WSL, was buggy and had some issues with filesystem performance i think. I prefer having a 100% real linux & windows OSes that just work over slightly buggy workarounds. Plus i also host a lot of homeserver services (media server, SMB server, postgres db etc) on the linux host and its cleaner having those run on the host. If i want to do something very demanding on the host for example I can shutdown the VM.
- hutrdvnj 5y ago
  Wouldn't it be easier to have 2 devices, one specifically designed for gaming and one running 24/7 for home server services. A cheap used Thinkpad could be used as linux device so it doesn't have to be expensive. Plus if you value your own time and effort you have to put in for your current setup, then it might actually be cheaper.
  - pizza234 5y ago
    It depends on what do you mean with "easier". Technically speaking, for machines where VFIO works, there is no maintenance, and the setup can be easy (excluding one significant issue due to a specific linux kernel upgrade, my VFIO setup procedure has been essentially the same for years, on multiple machines). So, assuming that VFIO setup setup/maintenance is very easy, having a second machine is just redundant. One convenience not to forget is safety. If a Windows VM gets infected for whatever reasons, rolling back the system is performed by literally deleting one file.
  - z3t4 5y ago
    You basically just need an extra gfx card (or even headless host). And why would you not want a fast Linux OS? Many SSD's and lots of RAM to make the IDE fast. Powerful CPU for compiling. Good GFX for better latency/hz.

porker 5y ago

This has been my dream setup for years, so that rather than a Windows host with Linux VMs I can run the other way round, enable ZFS and continually snapshot the Windows VM, and avoid the usual Windows failure as bits flip or sectors go bad.

Cyph0n 5y ago

Same setup here! I am running Proxmox on the host to streamline managing VMs and storage. Proxmox comes with a nice web GUI, which makes it very east to monitor system state. I have a Windows VM for gaming that owns the single Nvidia GPU. I also have a few Linux VMs for development (via VS Code remote) and media management. As far as storage goes, I don’t have anything too fancy. Proxmox is installed on an SSD. I have a second SSD for VM images. For all other storage (media, photos, VM image backups, etc.), I have a 3 disk ZFS pool consisting of a single RAID-Z1 vdev - yea, it’s risky, but losing the pool wouldn’t be the end of the world. One of the cool things about this kind of setup is being able to easily restore VMs from backup. Some time back, I accidentally screwed up my Windows install by enabling Hyper-V (nested virt). I panicked at first, but then remembered that I have daily snapshots of the VM. I had it back up and running within 10 minutes :) All in all, it was fun to setup and has been running very smoothly.

zaptheimpaler 5y ago
Yes I love that an entire VM is basically 2 files - one disk image, one libvirt config. Makes backup and restore so easy :)
j1elo 5y ago
Would you mind talking a bit about the small decisions you took for a Proxmox setup? I am slowly learning and planning my (very)small server setup. Things like: * Are the VM image backups you mentioned, done to your pool by means of the ZFS snapshots? Or done at the file-level with rsync or similar. * Do you make backups of the Proxmox installation? Similar as before: is Proxmox itself on a ZFS volume, so backups can be done just by doing ZFS snapshots? The installer lets me choose between an LVM-Thin + ext4, or a ZFS filesystem, and I was wondering whether to choose one or the other, for maximum convenience. * "Proxmox is installed on an SSD": isn't that a bit wasteful? I mean, doesn't the Proxmox system just take like 1 or 2 GB at most? I have a Lenovo ThinkCentre m910q which brings a 160GB M.2 NVMe disk, and another 320GB SSD disk... so I an in the process of deciding where to put each thing. Although for bigger storage I'm also considering if adding a 1 or 2 TB USB3 external disk would make sense (to store user backups like photos, documents, and also for the server's system backups)

mschuster91 5y ago

Which GPU do you have? I tried using a Mac Pro 5.1 as host, Debian Testing as OS and various Windows varieties as well as Linux (again Debian Testing) as guest - but I always, always run into that nasty PCI reset bug (https://www.nicksherlock.com/2020/11/working-around-the-amd-gpu-reset-bug-on-proxmox/ https://www.nicksherlock.com/2020/11/working-around-the-amd-...) :(

MegaDeKay 5y ago
Are you saying that that workaround doesn't help you? You could drop into their Discord on the vendor-reset channel and see if they can help you out. https://discord.gg/FJs9ufyu https://discord.gg/FJs9ufyu

isatty 5y ago

I use a vfio setup as well but with two GPUs (one integrated, one dedicated) and hotswap the dedicated GPU between the two whenever I’m playing games (I play on both Linux and Windows).

keyb0ardninja 5y ago
Is it possible to switch the GPU while the host is running, that is without rebooting?
- isatty 5y ago
  It is, see: https://stty.io/2018/07/27/running-windows-with-pci-passthrough-on-gentoo-linux/ https://stty.io/2018/07/27/running-windows-with-pci-passthro... (Relevant scripts sub-heading)
  - keyb0ardninja 5y ago
    Thanks a lot, I'll check it out.

rbanffy 5y ago

If you have more than one GPU (even as in one on-board and one discrete) you can, with some effort, dedicate one of them to a VM and let the host use the other. Low-yield crypto mining is a common usage for that kind of arrangement, but I suppose VMs would work just fine too. Or go full Unix and attach a serial terminal to the console port and happy linuxing from there while you ignore the built-in gaming console running off a VM. This is more or less what every modern console does anyway - a hypervisor under everything. ;-)

chevill 5y ago

> most online games with anti-cheat etc. Any notable exceptions? What's the performance like compared to the same games in a native Windows install. I tried proton several months ago and I had to go back to Windows because it wasn't even close to good enough as far as compatability goes. I'd really love to abandon Windows as a desktop OS and just game in a VM though.

Fabricio20 5y ago
A pretty notable example is probably BattlEye, which is used by several major FPS games [1] like Tarkov, Rainbow Six and PUBG. They banned the use of virtual machines [2] back in 2020 and their AC solution does not work on Linux. Valve seems to be working on it [3], though their solution might be tailored to their SteamDeck hardware instead of a generic proton fix. [1]: https://www.battleye.com/ https://www.battleye.com/ [2]: https://twitter.com/TheBattlEye/status/1289027672186720263?s=20 https://twitter.com/TheBattlEye/status/1289027672186720263?s... [3]: https://partner.steamgames.com/doc/steamdeck/faq https://partner.steamgames.com/doc/steamdeck/faq
- Nullabillity 5y ago
  PUBG uses EasyAntiCheat these days, rather than BattlEye.
deleted 5y ago
[deleted]

moondev 5y ago

I run the same setup but with esxi as the host OS. It's quite a bit easier to setup since esxi already runs headless by design, as well as easy PCI passthrough configuration via the host client html5 UI

MegaDeKay 5y ago

I have a VFIO setup as well but with dual GPUs. As you say, it is pretty awesome. I keep setting up other OS's for fun. I have an old nVidia NVS300 card that is supported natively under macOS, and also fired up a WindowsXP setup for grins. There were some interesting snags in getting each one going, and I learned a lot along the way. The Arch wiki is indeed spectacular.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

EchoReflection 5y ago

@zaptheimpaler thank you for sharing this.