And you can't easily switch between several virtualized machines like that. Windows will bluescreen if you remove a graphics card while in use, and I'm not sure

And you can't easily switch between several virtualized machines like that. Windows will bluescreen if you remove a graphics card while in use, and I'm not sure OSX/Linux guests will fare better. Passthrough only really works reliably if you only have one guest active at a time and never try to use the graphics card from the host.