9 ms·
With Windows 10 doing things like this, and Windows 7 being aggressively deprecated, I decided I finally wanted to ditch Windows as my daily driver. It was goo
by fpgaminer 11y ago
With Windows 10 doing things like this, and Windows 7 being aggressively deprecated, I decided I finally wanted to ditch Windows as my daily driver. It was good timing, then, that I watched the "7 Gamers 1 CPU" video from LinuxTechTips, which showed me how to do it without sacrificing gaming.
Most everything I use runs in Linux, and for the few things that don't I can spin up a VM. But there's the caveat that is as old as Windows itself ... gaming. Dual booting isn't an option for me, because I tend to run various server VMs 24/7. I also didn't want a second tower just for Windows; that's non-ideal. But wait! Apparently modern CPUs have this fancy thing called IOMMU I never knew about, which lets you give direct access to PCI-E devices, like GPUs, to a VM.
So I bought up a 5820K, a X99 motherboard, and an extra graphics card, and away I went. I now have Arch Linux running as the host system where I do all my work. Windows 10 sits in a VM where it's given direct and exclusive access to a GTX 970 for graphics. So far, things are working well and Windows runs the benchmark apps just fine.
The end result is that I can use Linux as my daily driver, but flip over to my Windows VM whenever I want to game. It keeps Windows isolated, which is great from a privacy/security perspective, I don't have to dual-boot, and best of all I can spin up different VMs if I want to use the beastly GTX 970 for other work like machine learning. Thanks to using the 5820K which has a few Xeon features in it, the entire process when smoothly. The biggest caveat is I haven't found a DisplayPort switch yet, so I can't hook up the Windows VM to my primary Cinema monitor, and I haven't nailed down the mouse+keyboard thing yet. QEMU's virtual mouse+keyboard doesn't normally work with these kinds of VMs, so you have to hack it in, and even then it likes to randomly stop working. I could just grab a second set of physical mouse+keyboard but that's not ideal. Steam Steaming will probably be the best option for now.
- hatsix 11y agoIs there a write-up on this that you can recommend?
- zymhan 11y agoThe Arch docs are super helpful on this, even if you're not running Arch: https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#vfio-pci https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVM...
- abledon 11y agoDo you have a link to a recommended tutorial?
- fpgaminer 11y agoThe VFIO blog was the best resource I found. Start at the first article (http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-1-hardware.html http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part...) in the how to series, work through that, and then read all the other posts on there. It's also helpful to use https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVM... , as that covers some of the things that the VFIO blog glosses over and it's a little more up-to-date. This is all bleeding edge stuff. For example, VFIO broke in a recent Linux kernel and only got fixed and released in Arch's repos a few days before I started my installation :P
- sofaofthedamned 11y agoAgreed, this the same info I followed to get Elite Dangerous working in a Windows VM on a Ubuntu host. Feels very nice to have Windows not as the root of a machine.
- deleted 11y ago[deleted]
- melloclello 11y agoHave you considered: you don't really need to play video games anyway
- Desustorm 11y agoNor do you need to be on Hacker News...
- fredley 11y agoYou do need to relax and unwind though, and video games are a great way to do that.
- omaranto 11y agoIt's not something that requires explicit consideration: probably anybody you ask will tell you that no, they don't need to play videogames; people who play them normally do it because they enjoy it, not because they need to. (It feels weird to have to explain that.)
- wongarsu 11y agoRight, who needs hobbys or fun anyway. /s
- xj9 11y agoYou don't really need to do anything that doesn't provide for your economic well-being. Some things a just for fun and that's great!
- id 11y agoYou don't need to provide for your economic well-being either.
- ska 11y agoHave you considered pausing before hitting "reply" and asking yourself: "does this comment add anything? does it have anything intelligent to say?"
- HCIdivision17 11y ago
- joshAg 11y agoHow much of a performance hit do you get running a VM with vtx and vtd versus running windows bare metal? Is it as simple as it seems? I'd love to switch over to a VM for windows games backed by vtx and vtd, but I've been worried that it'd be waay more work.
- swinglock 11y agoKeep in mind there's more to it than pure frames per second. You also got latency and jitter to worry about which is vital but almost always ignored, probably because it's difficult to measure and understand. Doing it properly still required special hardware and software I believe.
- fpgaminer 11y agoIndeed, I've seen some talk about latency/jitter issues on the vfio mailing list. I haven't tested it myself yet, so we'll see how it goes. Though I figure things will just naturally get better over time as the kernel's support improves.
- miffe 11y agoI tried this a while back. My results can be seen at http://miffe.org/bench/ http://miffe.org/bench/ The largest problem is that the NVIDIA driver refuses to work when it detects that it's in a VM. The workaround is to hide the VM status for the guest os, by using -cpu host,kvm=off and disabling the tweaks that makes windows work better (hv_time,hv_relaxed,hv_vapic).
- Diederich 11y agoIf you made a nice blog post about this, in detail, and shared it, a lot of folks would find it extremely interesting!
- fpgaminer 11y agoWow, that's a great resource! I'm glad to see your disk benchmarks were also all over the place. I couldn't seem to make much sense of CrystalMark's results, even when increasing the test size above my RAM size to eliminate caching. I'm using virtio to an SSD dedicated to the VM and saw, like you, some markers were better in the VM than they should be, and some where significantly worse. Very weird. Luckily I won't matter for gaming.
- littletinman 11y agoAny chance you have a parts list for your build?
- MichaelApproved 11y agoNot OP but, if you're interested in building your own PC and need some guidance, check out the Build A PC reddit. They're extremely helpful and knowledgeable in there. https://www.reddit.com/r/buildapc https://www.reddit.com/r/buildapc Another similar resource is overclocking https://www.reddit.com/r/overclocking https://www.reddit.com/r/overclocking
- fpgaminer 11y agoMy build: http://pcpartpicker.com/p/JVsKYJ http://pcpartpicker.com/p/JVsKYJ I splurged on an M.2 PCI-E drive. 2 GB/s read!? How could I turn that down? No way I'll ever use that drive to its full potential, but I love the M.2 form factor. Only thing I'd change is using AIO cooling for the CPU and the S340 instead of the H440 case. I had to use the H440 for the air cooler, and that was not an aesthetically pleasing choice. The H440 also has a number of obnoxious problems.
- yegle 11y agoI also want to know more about this, care to share it? Thanks!
- DanGarcia595 11y agoIt would be amazing if you could write a separate post with further details. I think many people here would love to know more.
- zymhan 11y agoThere are many guides about this, just search for something like "VGA PCI Passthrough" https://www.google.com/search?safe=off&q=VGA+PCI+Passthrough& https://www.google.com/search?safe=off&q=VGA+PCI+Passthrough...
- siegecraft 11y agoFor anyone that tries this route, following random tutorials on VGA PCI Passthrough is a route to sure frustration and time wasting (I tried it ~2 or 3 years ago, whenever I built my previous PC). You have to make sure your video card, motherboard chipset, audio, and anything else you want to pass through are all supported. It's much simpler if you can just follow a list that says "this combination of hardware + software works perfectly."
- zymhan 11y agoAll I did was check the Wikipedia page [1] for IOMMU-supporting hardware, and I needed to upgrade my CPU from a Phenom to FX. You don't need special audio, just the right motherboard (chipset specifically) and CPU. If you do PCI passthrough, it doesn't matter if the Linux host can talk to the card, the guest OS will take care of everything. Otherwise, check the forums. This has been done many times over the years, and there's no shortage of existing information. [1] https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardw...
- sp4ke 11y agoHolly crap, I always dreamed to achieve what you did and never knew direct GPU access was possible, even though I used Qemu for work before. Right now I am basically running a windows pc for gaming and a linux laptop for work. It seems IOMMU is compatible with all my gaming tower, and with many modern cpus/chipsets. I will make a test over a live CD anb if it works migrate my whole pc and blog about it. Thanks very much for the ida
- SeriousM 11y agoWhat's your blog url?
- zymhan 11y agoI've done this too! I was able to do it with my AMD FX-8350 and GA-990FX-UD3, as they both support IOMMU. Passing through my R9 390 to the Windows guest, and the only issue I've encountered the whole time was having to disable hardware acceleration for flash videos. The whole KVM/Qemu ecosystem is great.
- throwaway3349 11y agopiggybacking on top comment. I have yet to work through this process, but there's this write-up for doing this in Ubuntu I found a few weeks ago: http://ubuntuforums.org/showthread.php?t=2266916 http://ubuntuforums.org/showthread.php?t=2266916
- JoshuaRedmond 11y agoThe link to the video by LinusTechTips for anyone interested[0]. He actually had a heck of a time getting it working fully with all 7 graphics cards, as shutting a VM down wouldn't correctly release the card's resources (apparently an AMD-specific issue, but they denied it[1]), causing him to have to do a full reboot of the whole system every time one VM needed a restart[1]. [0] - https://www.youtube.com/watch?v=LXOaCkbt4lI https://www.youtube.com/watch?v=LXOaCkbt4lI (14:57) [1] - https://www.youtube.com/watch?v=lThX7UovKCc https://www.youtube.com/watch?v=lThX7UovKCc (1:35:15 - really long stream of him disassembling it live, so forgive me for not finding exactly where he said each thing).
- elFarto 11y agoHere's the link to where he talks about AMD on that stream https://youtu.be/lThX7UovKCc?t=34m https://youtu.be/lThX7UovKCc?t=34m
- Namidairo 11y agoSounds suspiciously like this hardware quirk[0] that was only supposed to affect Hawaii/Bonaire... Guess it's a little more widespread. Easily fixed, I guess[1]. Then again Nvidia cards have their own set of horrible problems too, although theirs are somewhat on the side of malicious intent rather than incompetence. [0] - https://github.com/qemu/qemu/blob/master/hw/vfio/pci-quirks.c#L1049-L1069 https://github.com/qemu/qemu/blob/master/hw/vfio/pci-quirks.... [1] - https://github.com/qemu/qemu/blob/master/hw/vfio/pci-quirks.c#L1179-L1202 https://github.com/qemu/qemu/blob/master/hw/vfio/pci-quirks....
- Karunamon 11y agoThis was the last straw for me. My current chipset doesn't support the VT-D stuff, so it's an excuse to shop around for an upgrade. Figure I get a board that supports the integrated graphics, run Linux on that, and then pass through my GTX980 to a VM.
- goda90 11y agoHave your tried Synergy for your mouse/keyboard situation? Another possibility is perhaps a KVM switch that if you plug in two outputs into the same computer that it is recognized as two distinct USB devices. I don't know if it's possible but then you could pass through one to the VM directly.
- MatthaeusHarris 11y agoSynergy is decidedly non-optimal for gaming, unfortunately. Various games try to grab the mouse pointer in different ways and synergy throws a wrench into at least some of them.
- blibble 11y agoyou can bind a key to switch to relative mode. since switching to VM based gaming about 6 months ago I've played and completed: SC2 LOTV, Bioshock Infinite, Crysis 1/2, and The Talos Principle and have not had a single problem.
- giancarlostoro 11y agoThank you very much for this post, this had been one of the things I've wanted to do for so many years but never knew if anyone else found a proper way of doing it, every time I would google this I would end up with bad results.
- paler 11y agoI'm happy to hear that gaming works well, and apparently my decidedly older CPU (AMD Phenom X4 965) supports the technology as well, which is interesting to learn. However, my main reason for needing Windows is for music recording and mixing via Cubase and other DAW software. This requires really low-latency access to my USB and PCI sound cards. Does anyone have experience with running applications like this in a VM? Is it feasible? I would be thrilled if I could finally move to Linux or FreeBSD entirely and just keep a Windows 7 VM, all sandboxed up with no Internet access, just for the music recording.
- cmdrfred 11y agoI have found that a lot of the audio editing software you can get away with quite low specs. I record my podcasts on a 8 year old laptop running Windows XP. It rarely connects to the internet and I record to a thumb drive that is backed up so there is nothing to lose on it. In this particular case something similar might be the best option. For pod-casting anything over 2 or 3 ms in my headphones throws me off my game.
- plaguuuuuu 11y agoUnlike podcasts, music recording and mixing is extremely CPU intensive. Especially if you run a lot of analog simulating synths and stuff.
- cmdrfred 11y agoI stand corrected.
- jsolson 11y agoVirtual CPUs with VTx can be made to run at essentially native speed for most workloads on most fairly modern CPUs. Memory performance can be subpar, although there's public documentation on techniques that can improve that: https://help.ubuntu.com/community/KVM%20-%20Using%20Hugepages https://help.ubuntu.com/community/KVM%20-%20Using%20Hugepage... I/O performance is a whole other story, dependent on many workload and hypervisor specific factors.
- TheGuyWhoCodes 11y agoIt's so nice for KVM and Xen to support this. I'd wish virtualbox would support pci pass-through. Virtualbox being the default and most used vm with vagrant it would be awesome to spin up a linux vm with pci pass-through, so you can have a machine learning vm with cuda.
- miffe 11y agoVirtualBox does support PCI passthrough. See https://www.virtualbox.org/manual/ch09.html#pcipassthrough https://www.virtualbox.org/manual/ch09.html#pcipassthrough
- TheGuyWhoCodes 11y agoI have never seen someone who actually got it to work. I read somewhere where it might be possible with a Linux host and a Linux guest. If anyone was ever successful with a windows host and a Linux guest I'd love to read how...
- mc32 11y agoYeah this is not a good move; thankfully for now it can be turned off as explained in the text of the article.
- zanny 11y agoJust general disclosure that Steam on Linux does have 1600 games now, and at least a few dozen AAA titles. I have no problem gaming on Linux like any console owner would - if it releases for my platform, awesome, new games. If it does not, it might as well not exist to me. I have no shortage of games to play (I still have to beat Witcher 2 and I'm 20 hours in... 3 years later). And Blizzard stuff / anything older than 2010 runs perfectly in Wine nowadays, and with Gallium Nine I can play Skyrim on my 290 at its max DX9 settings at ~100 fps.
- y4mi 11y ago1600 "games". most shouldn't be called such because of their quality. vanilla skyrim never required a good computer... its a different story if it's fully modded, but i doubt you've done that, as you'd be pushing 30 fps with a 290 in that case
- ionised 11y agoI just bit the bullet and kept my gaming machine on Windows 10 and built a whole new Debian machine for all my dev work and personal computing. I would prefer to drop Windows entirely though and was looking forward to SteamOS making strides in this area but it doesn't look too promising. WINE solves a lot of problems but it's not perfect, and a lot of games still don't work with it according to its compatibility database.
- blahblah12 11y agoSeriously did people even click the link? This is not some insidious full screen advertisement . . . it's windows spotlight. You were allowing Bing to show you random photos by enabling this feature. If you didn't want that, then disable it . . . give a break. Ads on installers have been common for ages. Ubuntu shows advertisements . . . this is out of control with the fear mongering.
- unprepare 11y agoBut the enabling by default of advertisements in the OS is certainly something I'm not comfortable with, no matter how pretty the ad is or how simple it is to disable.
- winter_blue 11y agoI would really appreciate a write-up, or link to a step-by-step guide on this as well. I have a 5820K as well, with a GA-X99-UD3P motherboard and 32 gigs of quad-channel DDR4 RAM, and I dual-boot Linux and Windows. Windows just has Steam, and I do all my work on Linux. It would be wonderful to not have to reboot and switches OSes and just run Windows off a VM with full graphics performance! :)