14 ms·

Vgpu_unlock: Unlock vGPU functionality for consumer grade GPUs

468 points by fragileone 6y ago

h2odragon 6y ago
> In order to make these checks pass the hooks in vgpu_unlock_hooks.c will look for a ioremap call that maps the physical address range that contain the magic and key values, recalculate the addresses of those values into the virtual address space of the kernel module, monitor memcpy operations reading at those addresses, and if such an operation occurs, keep a copy of the value until both are known, locate the lookup tables in the .rodata section of nv-kernel.o, find the signature and data bocks, validate the signature, decrypt the blocks, edit the PCI device ID in the decrypted data, reencrypt the blocks, regenerate the signature and insert the magic, blocks and signature into the table of vGPU capable magic values. And that's what they do. I'm very grateful I wasn't required to figure that out.
- stingraycharles 6y ago
  I love the conciseness of this explanation. In just a few sentences, I completely understand the solution, but at the same time also understand the black magic wizardry that was required to pull it off.
  - jacquesm 6y ago
    Not to mention the many hours or days of being stumped. This sort of victory typically doesn't happen overnight. What bugs me about companies like NV is that if they just sold their hardware and published the specs they'd probably sell more than with all this ridiculously locked down nonsense, it is just a lot of work thrown at limiting your customers and protecting a broken business model.
    - eli 6y ago
      But they’d also sells fewer high end models. I don’t doubt that they’ve done the math.
    - TomVDB 6y ago
      Is a “broken business model” one that requires you to pay for extra additional features? If Nvidia enabled all their professional features on all gaming SKUs, the only reason to buy a professional SKU would be additional memory. Today, they make almost $1B per year in the professional non-datacenter business alone. There is no way they’d be able to compensate that revenue with volume (and gross margins would obviously tank as well, which makes Wall Street very unhappy.) That’s obviously even more so in today’s market conditions. Do you feel it’s justified that you have to pay $10K extra for the self-driving feature on a Tesla? Or should they also be forced to give away that feature for free? After all, it’s just a SW upgrade. (Don’t mistake this for an endorsement...)
      - lawl 6y ago
        > Do you feel it’s justified that you have to pay $10K extra for the self-driving feature on a Tesla? Or should they also be forced to give away that feature for free? After all, it’s just a SW upgrade. I feel like i already paid for the hardware. If telsa says it's cheaper for them to stick the necessary hardware into every car, i'm still paying for it if i buy one without self-driving. Thus if i think the tesla software isn't worth 10k and i'd rather use openpilot, i feel like i should have the right to do that. But nvidia is also actively interfering with open source drivers (nouveau) with signature checks etc.
        xvilka 6y ago
        Yes, NVIDIA is actively malicious on that front.
        TomVDB 6y ago
        (Leaving aside that Tesla also actively prevents you from running your own software on its platform...) The whole focus on hardware is just bizarre. When you buy a piece of SW that has free features but requires a license key to unlock advanced features, everything is fine, but the moment HW is involved all of that flies out of the window. Extra features cost money to implement. Companies want to be paid for it. A company like Nvidia could decide to make 2 pieces of silicon, one with professional features and once without. Or they could disable it. Obviously, you’d prefer the first option, even if it absolutely makes no sense to do so. It’d be a waste of engineering resources that could have been spent on future products. Deciding to disable a feature on a piece of silicon is no different than changing a #define or adding “if (option)” to disable an advanced feature. By doing so, I have the option to not pay for an advanced feature that I don’t need. I don’t want the self-driving option in a Tesla and I’m very happy to have that option.
    - oatmeal_croc 6y ago
      I have a feeling that they've done the math and have realized what makes them the most money.
    - matheusmoreira 6y ago
      Their business model is not broken. Not yet. With hardware unlocking software like vgpu_unlock, we can break it.
      - throwaway2048 6y ago
        Count on driver updates breaking this workaround
        matheusmoreira 6y ago
        If they break it, the people who really need this feature will simply not upgrade. Companies run decades old software all the time, this isn't going to be any different. It's just like nvidia's ridiculous blocking of cryptocurrency mining workloads. Once the fix is out, it's over. Also I have no doubt people will find other ways to unlock the hardware.
        throwaway2048 6y ago
        stuff like video games often require driver updates to function, which is a major use case of this hack. Not to mention older nvidia drivers do not support newer linux kernels.
        webmaven 6y ago
        > Count on driver updates breaking this workaround If the workaround results in enough money being left on the table, this might prompt 3rd party investment in open source drivers in order to keep the workaround available by eliminating the dependence on Nvidia's proprietary drivers.
    - PragmaticPulp 6y ago
      > they'd probably sell more than with all this ridiculously locked down nonsense It's currently impossible to find any nVidia GPU in stock because the demand far outstrips the supply. Market segmentation is only helping their profit margins, not hurting it.
    - deleted 6y ago
      [deleted]
- kbumsik 6y ago
  Also, I wonder what kinds of skills are required to figure this out? I don't think just knowing Linux kernel internals would be enough.
  - Sebb767 6y ago
    The actual "trick" behind this is well known and has been done for quite some time. One could actually solder in a different device id in the GTX 6xx series [0] or flash a different VBIOS on the GTX 5xx ones. The real achievement here is implementing this in software without touching the GPU. This is not to downplay the OP, of course - this is truly great and I'm sure it was a lot of work. But the hardware part is not new. [0] https://web.archive.org/web/20200814064418/https://www.eevblog.com/forum/general-computing/hacking-nvidia-cards-into-their-professional-counterparts/ https://web.archive.org/web/20200814064418/https://www.eevbl...
ur-whale 6y ago
If - like me - you don't have a clue what vGPU is: https://www.nvidia.com/en-us/data-center/virtual-solutions/ https://www.nvidia.com/en-us/data-center/virtual-solutions/ TL;DR: seems to be something useful for deploying GPUs in the cloud, but I may not have understood fully.
- lovedswain 6y ago
  It instantiates multiple logical PCI adaptors for a single physical adaptor. The logical adaptors can then be mapped into VMs which can directly program a hardware-virtualized view of the graphics card. Intel has the same feature in their graphics and networking chips
  - ur-whale 6y ago
    Thanks for the explanation, but that's more of a "this is how it works" than a "this is why it's useful". What would be the main use case?
    - wmf 6y ago
      The use case is allowing the host system and VM(s) to access the same GPU at the same time.
      - ur-whale 6y ago
        Yeah, I got that from the technical explanation. What's the practical use case, as in, when would I need this? [EDIT]: To maybe ask a better way: will this practically help me train my DNN faster? Or if I'm a cloud vendor, will this allow me to deploy cheaper GPU for my users? I guess I'm asking about the economic value of the hack.
        lovedswain 6y ago
        Running certain ML models in VMs Running CUDA in VMs Running transcoders in VMs Running <anything that needs a GPU> in VMs
sudosysgen 6y ago
Amazing! Simply amazing! This not only enables the use of GPGPU on VMs, but also enables the use of a single GPU to virtualize Windows video games from Linux! This means that one of the major problems with Linux on the desktop for power users goes away, and it also means that we can now deploy Linux only GPU tech such as HIP on any operating system that supports this trick!
- ur-whale 6y ago
  > Amazing! Simply amazing! If it's such a cool feature, why does NVidia lock it away non-Tesla H/W? [EDIT]: Funny, but the answers to this question actually provide way better answers to the other question I posted in this thread (as in: what is this for).
  - matheusmoreira 6y ago
    To make people pay more.
    - deleted 6y ago
      [deleted]
  - sudosysgen 6y ago
    Because otherwise, people would be able to use non-Tesla GPUs for cloud compute workloads, drastically reducing the cost of cloud GPU compute, and it would also enable the use of non-Tesla GPUs as local GPGPU clusters - additionally reducing workstation GPU sales due to more efficient resource use. GPUs are a duopoly due to intellectual property laws and high costs of entry (the only companies I know of that are willing to compute are Chinese and only a result of sanctions), so for NVidia this just allows for more profit.
    - userbinator 6y ago
      Interestingly, Intel is probably the most open with its GPUs, although it wasn't always that way; perhaps they realised they couldn't compete on performance alone.
liuliu 6y ago
One thing I want to figure out (because I don't have a dedicated Windows gaming desktop), and the documentation on the internet seems sparse: it is my understanding that if I want to use PCIe passthrough with Windows VM, these GPUs cannot be available to the host machine at all, or technically it can, but I need to do some scripting to make sure the NVIDIA driver doesn't own these PCIe lanes before open Windows VM and re-enable it after shutdown? If I go with vGPU solution, I don't need to turn on / off NVIDIA driver for these PCIe lanes when running Windows VM? (I won't use these GPUs on host machine for display).
- matheusmoreira 6y ago
  Exactly. With GPU virtualization the driver is able to share the GPU resources with multiple systems such as the host operating system and guest virtual machine. Shame on nvidia for arbitrarily locking us out of this feature.
  - judge2020 6y ago
    Is there info on this sort of usage? I'd love to use the host for NVENC and a VM guest for traditional GPU stuff, but haven't been able to find anything on doing that.
  - liuliu 6y ago
    Got some time to try this now. It worked as expected, I have vgpu_vfio. However, it doesn't perfectly fit my needs. Particularly, my host system is "heavy", I need it to run CUDA etc, while the VM just to run games. However, it seems the 460.32.04 driver on host doesn't have full functionality, hence, cannot run CUDA on the host any more.
- deleted 6y ago
  [deleted]
- Youden 6y ago
  > One thing I want to figure out (because I don't have a dedicated Windows gaming desktop), and the documentation on the internet seems sparse: it is my understanding that if I want to use PCIe passthrough with Windows VM, these GPUs cannot be available to the host machine at all, or technically it can, but I need to do some scripting to make sure the NVIDIA driver doesn't own these PCIe lanes before open Windows VM and re-enable it after shutdown? The latter statement is correct. The GPU can be attached to the host but it has to be detached from the host before the VM starts using it. You may also need to get a dump of the GPU ROM and configure your VM to load it at start up. Regarding the script, mine resembles [0]. You need to remove the NVIDIA drivers and then attach the card to VFIO. And then the opposite afterwards. You may also need to image your GPU ROM [1] [0]: https://techblog.jeppson.org/2019/10/primary-vga-passthrough-in-proxmox/ https://techblog.jeppson.org/2019/10/primary-vga-passthrough... [1]: https://clayfreeman.github.io/gpu-passthrough/#imaging-the-gpu-rom https://clayfreeman.github.io/gpu-passthrough/#imaging-the-g...
neatze 6y ago
To me this is laughably naive question, but I ask it any way. My understanding is that CPU/GPU per application can make only single draw call in sequential manner. (eg. CPU->GPU->CPU->GPU) Could vgpu's be used for concurrent draw calls from multiple processes of an single application ?
- milkey_mouse 6y ago
  > My understanding is that CPU/GPU per application can make only single draw call in sequential manner. The limitation you're probably thinking of is in the OpenGL drivers/API, not in the GPU driver itself. OpenGL has global (per-application) state that needs to be tracked, so outside of a few special cases like texture uploading you have to only issue OpenGL calls from one thread. If applications use the lower-level Vulkan API, they can use a separate "command queue" for each thread. Both of those are graphics APIs, I'm less familiar with the compute-focused ones but I'm sure they can also process calls from multiple threads.
  - milkey_mouse 6y ago
    And VGPUS are isolated from one another, that's the whole point-so using multiple in one application would be very difficult, as I don't think they can share data/memory in any way.
  - neatze 6y ago
    My primitive thoughts: Threaded Computation on CPU -> Single GPU Call -> Parallel Computation on GPU -> Threaded Computation on CPU ... I wonder if it can be used in such way: Asyc Concurrent Computation on CPU -> Asyc Concurrent GPU Calls -> Parallel Time Independent Computations on GPU -> Asyc Concurrent Computation on CPU
- deleted 6y ago
  [deleted]
shmerl 6y ago
Is this for SR-IOV? It's too bad SR-IOV isn't supported on regular desktop AMD GPUs for example in the Linux driver.
- Nullabillity 6y ago
  Yes, this is basically NVidia's SR-IOV.
noodlesUK 6y ago
I wish Nvidia would open this up properly. The fact that intel integrated gpus can do GVT-G and I literally can’t buy a laptop which will do vgpu passthrough with an Nvidia card for any amount of money is infuriating.
- my123 6y ago
  GVT-g is gone on 10th-gen GPUs (Ice Lake) and later. Not supported on Intel dGPUs either.
  - cromka 6y ago
    Oh wow. Had no idea and I was explicitly planning my next server to be Intel because of the GVT-g. Why did they abandon it?
    - deleted 6y ago
      [deleted]
    - effie 6y ago
      More like did not prioritize it. https://github.com/intel/gvt-linux/issues/126 https://github.com/intel/gvt-linux/issues/126
WrtCdEvrydy 6y ago
This is a dumb question, but which hypervisor configuration is this targeted towards to. There's a lot of detail on the link which I appreciate but maybe I missed it.
- effie 6y ago
  KVM, maybe also Xen.
airocker 6y ago
This is super! What would it take to abstract it similar to CPU/Memory by specifying limits only in croups? Limits could be like GPU Memory size/amount of parallelization?
minimalist 6y ago
Related but different: - nvidia-patch [0] "This patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs." - About a week ago "NVIDIA Now Allows GeForce GPU Pass-Through For Windows VMs On Linux" [1]. Note, this is only for the driver on Windows VM guests not GNU/Linux guests. Hopefully the project in the OP will mean that GPU access is finally possible on GNU/Linux guests on Xen, thank you for sharing OP. [0]: https://github.com/keylase/nvidia-patch https://github.com/keylase/nvidia-patch [1]: https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-GeForce-Virt-On-Linux https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-G...
madjam002 6y ago
I built an "X gamers/workstations 1 CPU" type-build last year and this has been the main problem, I have two GPUs, one of which is super old and I have to choose which one I want to use when I boot up a VM. Will definitely be checking this out!
DCKing 6y ago
Dual booting is for chumps. If I could run a base Linux system and arbitrarily run fully hardware accelerated VMs of multiple Linux distros, BSDs and Windows, I'd be all over that. I could pretend here that I really need the ability to quickly switch between OSes, that I'd like VM-based snapshots, or that I have big use cases to multiplex the hardware power in my desktop box like that. I really don't need it. I just want it. I really hope Intel sees this as an opportunity for their DG2 graphics cards due out later this year. If anyone from Intel is reading this: if you guys want to carve out a niche for yourself, and have power users advocate for your hardware - this is it. Enable SR-IOV for your upcoming Xe DG2 GPU line just as you do for your Xe integrated graphics. Just observe the lengths that people go to for their Nvidia cards, injecting code into their proprietary drivers just to run this. You can make this a champion feature just by not disabling something your hardware can already do. Add some driver support for it in the mix and you'll have an instant enthusiast fanbase for years to come.
- strstr 6y ago
  Passthrough is workable right now. It’s a pain to get set up, but it is workable. You don’t need vgpu to get the job done. I’ve had two set ups over time: one based on a jank old secondary gpu that is used by the vm host, another based on just using the jank integrated graphics on my chip. Even still, I dual boot because it just works. It always works, and boot times are crazy low for Windows these days. No fighting with drivers. No fighting with latency issues for non-passthrough devices. It all just works.
  - DCKing 6y ago
    Oh I'm aware of passthrough. It's just a complete second class citizen because it isn't really virtualization, it's a hack. Virtualization is about multiplexing hardware. Passthrough is the opposite of multiplexing hardware: it's about yanking a peripheral from your host system and shoving it into one single guest VM. The fact that this yanking is poorly supported and has poor UX makes complete sense. I consider true peripheral multiplexing with true GPU virtualization to be the way of the future. It's true virtualization and doesn't even require you to sacrifice and/or babysit a single PCIe connected GPU. Passthrough is just a temporary hacky workaround that people have to apply now because there's nothing better. In the best case scenario - with hardware SR-IOV support plus basic driver support for it, enabling GPU access in your VM with SR-IOV would be a simple checkbox in the virtualization software of the host. GPU passthrough can't ever get there in terms of usability.
jarym 6y ago
Hacking at its finest! Nice
cercatrova 6y ago
For virtualized Windows from Linux, check out Looking Glass which I posted about previously https://news.ycombinator.com/item?id=22907306 https://news.ycombinator.com/item?id=22907306
albertzeyer 6y ago
The Python script actually mostly uses Frida (https://frida.re/ https://frida.re/) scripting. I haven't seen Frida before, but this looks very powerful. I did some similar (but very basic) things with GDB/LLDB scripting before but Frida seems to be done for exactly things like this.
ternox99 6y ago
I don't understand how and where I can download nvidia grid vgpu driver. Anyone can help me?
- liuliu 6y ago
  Register for evaluation: https://nvid.nvidia.com/dashboard/#/dashboard https://nvid.nvidia.com/dashboard/#/dashboard
  - ternox99 6y ago
    Thanks :)
williesleg 6y ago
Why can't these assholes post a list of compatible gpu's, why do I have to install this shit to see?
schaefer 6y ago
There's a lot of customer loyalty on the table waiting for the first GPU manufacturer to unlock this feature on consumer grade cards without forcing us to resort to hacks.
- deleted 6y ago
  [deleted]
- judge2020 6y ago
  Not many people are foregoing a GPU given this limitation, though, except for maybe miners (which will virtualize and pirate/never activate Windows if they really need Windows).
  - throwaway2048 6y ago
    that equilibrium only holds as long as your competitor doesn't offer it.
archi42 6y ago
This puts me in a tough spot: There are good reasons to go with nVidia (once they release GPUs with proper memory configuration), DLSS, RTX and now this. On the other hand, do I really want to give money to the company that locked this feature in the first place? Difficult call, but at least the ridiculous prices mean I can still think about it for a few more months.
- SXX 6y ago
  I'm very much AMD fanboy and I find RTX quite useless since it's only implemented in few games and work well in 2K only on really high-end cards. Yet AMD is no different there. They also locked SR-IOV for premium datacenter hardware only and they certainly want to keep some features away from consumers.
  - chaz6 6y ago
    Not particularly an nvidia fanboy, but RTX can also be used to speed up rendering in Blender with the E-Cycles add-on. https://blendermarket.com/products/e-cycles https://blendermarket.com/products/e-cycles
solnyshok 6y ago
what are those "some Geforce and Quadro GPUs that share the same physical chip as the Tesla GPUs"?
- broodbucket 6y ago
  it's in the source https://github.com/DualCoder/vgpu_unlock/blob/0675b563acdae8f6c5eb1a760fc0e5b853b1a63e/vgpu_unlock_hooks.c#L704 https://github.com/DualCoder/vgpu_unlock/blob/0675b563acdae8...
  - solnyshok 6y ago
    thanks! it is interesting. gtx 1060 3gb is ok, but rtx 3070 isn't. and it is time to upgrade from my trusty 970
Sebb767 6y ago
If anyone needs a list of currently supported CPUs, you can find it in the source code: https://github.com/DualCoder/vgpu_unlock/blob/master/vgpu_unlock#L97-L139 https://github.com/DualCoder/vgpu_unlock/blob/master/vgpu_un... (the comments behind the device ids)
opan 6y ago
Is this only for Nvidia GPUs? If so, why not put it in the title?
kkielhofner 6y ago
At the risk of piling on without value: "Amazing, simply amazing". I've been (more or less) accused of being an Nvidia fanboy on HN previously but this is an area where I've always thought Nvidia has their market segmentation wrong. Just wrong. This is great work. (Period, as in "end of sentence, mic drop").
effie 6y ago
Wow, does this mean NVIDIA consumer cards can now do multi-VM multiplexing of single GPU? On AMD side, only special cards like FirePro S7150 can do this. Does this work also on Xen? NVIDIA drivers were always non-functional with Xen+consumer cards.
effie 6y ago
The number of technology layers one must understand and control to make GPU draw what VM wants is frankly insane. Hypervisor channels, GPU driver code, graphics server API, graphics toolkit libraries - all these have several variants. It seems close to impossible to get anything done in this space. I just want to send drawing commands from a node over network to another node with a GPU. Like "draw a black rectangle 20,20,200,200 on main GPU on VM at 192.168.1.102". How would I do that in the simplest possible way? Is there some network graphics command protocol? Like X11, but simpler and faster, just the raw drawing commands.
pasnew 6y ago
Great work ! Does it limited by Nvidia's 90 day vGPU software evaluation ?
chern12 6y ago
Does this mean we'll now see miners make 3-5 vVGPUs from one 3090 for triple the hashrate per card? Because I'm already having trouble getting a 3090 as it stands.
rektide 6y ago
amd hasn't even been advertising their old "mxgpu technology" nor any follow up, & I am quite sad about it. hacks like nucleus coop[1] to run multiple instances of games on a desktop work via a shared screen & hacks, but I'd rather have a multi-head desktop with a bunch of gaming vms. [1] https://github.com/lucasassislar/nucleuscoop https://github.com/lucasassislar/nucleuscoop

468 points by fragileone 6y ago

> In order to make these checks pass the hooks in vgpu_unlock_hooks.c will look for a ioremap call that maps the physical address range that contain the magic and key values, recalculate the addresses of those values into the virtual address space of the kernel module, monitor memcpy operations reading at those addresses, and if such an operation occurs, keep a copy of the value until both are known, locate the lookup tables in the .rodata section of nv-kernel.o, find the signature and data bocks, validate the signature, decrypt the blocks, edit the PCI device ID in the decrypted data, reencrypt the blocks, regenerate the signature and insert the magic, blocks and signature into the table of vGPU capable magic values. And that's what they do. I'm very grateful I wasn't required to figure that out.

stingraycharles 6y ago
I love the conciseness of this explanation. In just a few sentences, I completely understand the solution, but at the same time also understand the black magic wizardry that was required to pull it off.
- jacquesm 6y ago
  Not to mention the many hours or days of being stumped. This sort of victory typically doesn't happen overnight. What bugs me about companies like NV is that if they just sold their hardware and published the specs they'd probably sell more than with all this ridiculously locked down nonsense, it is just a lot of work thrown at limiting your customers and protecting a broken business model.
  - eli 6y ago
    But they’d also sells fewer high end models. I don’t doubt that they’ve done the math.
  - TomVDB 6y ago
    Is a “broken business model” one that requires you to pay for extra additional features? If Nvidia enabled all their professional features on all gaming SKUs, the only reason to buy a professional SKU would be additional memory. Today, they make almost $1B per year in the professional non-datacenter business alone. There is no way they’d be able to compensate that revenue with volume (and gross margins would obviously tank as well, which makes Wall Street very unhappy.) That’s obviously even more so in today’s market conditions. Do you feel it’s justified that you have to pay $10K extra for the self-driving feature on a Tesla? Or should they also be forced to give away that feature for free? After all, it’s just a SW upgrade. (Don’t mistake this for an endorsement...)
    - lawl 6y ago
      > Do you feel it’s justified that you have to pay $10K extra for the self-driving feature on a Tesla? Or should they also be forced to give away that feature for free? After all, it’s just a SW upgrade. I feel like i already paid for the hardware. If telsa says it's cheaper for them to stick the necessary hardware into every car, i'm still paying for it if i buy one without self-driving. Thus if i think the tesla software isn't worth 10k and i'd rather use openpilot, i feel like i should have the right to do that. But nvidia is also actively interfering with open source drivers (nouveau) with signature checks etc.
      - xvilka 6y ago
        Yes, NVIDIA is actively malicious on that front.
      - TomVDB 6y ago
        (Leaving aside that Tesla also actively prevents you from running your own software on its platform...) The whole focus on hardware is just bizarre. When you buy a piece of SW that has free features but requires a license key to unlock advanced features, everything is fine, but the moment HW is involved all of that flies out of the window. Extra features cost money to implement. Companies want to be paid for it. A company like Nvidia could decide to make 2 pieces of silicon, one with professional features and once without. Or they could disable it. Obviously, you’d prefer the first option, even if it absolutely makes no sense to do so. It’d be a waste of engineering resources that could have been spent on future products. Deciding to disable a feature on a piece of silicon is no different than changing a #define or adding “if (option)” to disable an advanced feature. By doing so, I have the option to not pay for an advanced feature that I don’t need. I don’t want the self-driving option in a Tesla and I’m very happy to have that option.
  - oatmeal_croc 6y ago
    I have a feeling that they've done the math and have realized what makes them the most money.
  - matheusmoreira 6y ago
    Their business model is not broken. Not yet. With hardware unlocking software like vgpu_unlock, we can break it.
    - throwaway2048 6y ago
      Count on driver updates breaking this workaround
      - matheusmoreira 6y ago
        If they break it, the people who really need this feature will simply not upgrade. Companies run decades old software all the time, this isn't going to be any different. It's just like nvidia's ridiculous blocking of cryptocurrency mining workloads. Once the fix is out, it's over. Also I have no doubt people will find other ways to unlock the hardware.
        throwaway2048 6y ago
        stuff like video games often require driver updates to function, which is a major use case of this hack. Not to mention older nvidia drivers do not support newer linux kernels.
      - webmaven 6y ago
        > Count on driver updates breaking this workaround If the workaround results in enough money being left on the table, this might prompt 3rd party investment in open source drivers in order to keep the workaround available by eliminating the dependence on Nvidia's proprietary drivers.
  - PragmaticPulp 6y ago
    > they'd probably sell more than with all this ridiculously locked down nonsense It's currently impossible to find any nVidia GPU in stock because the demand far outstrips the supply. Market segmentation is only helping their profit margins, not hurting it.
  - deleted 6y ago
    [deleted]
kbumsik 6y ago
Also, I wonder what kinds of skills are required to figure this out? I don't think just knowing Linux kernel internals would be enough.
- Sebb767 6y ago
  The actual "trick" behind this is well known and has been done for quite some time. One could actually solder in a different device id in the GTX 6xx series [0] or flash a different VBIOS on the GTX 5xx ones. The real achievement here is implementing this in software without touching the GPU. This is not to downplay the OP, of course - this is truly great and I'm sure it was a lot of work. But the hardware part is not new. [0] https://web.archive.org/web/20200814064418/https://www.eevblog.com/forum/general-computing/hacking-nvidia-cards-into-their-professional-counterparts/ https://web.archive.org/web/20200814064418/https://www.eevbl...

ur-whale 6y ago

If - like me - you don't have a clue what vGPU is: https://www.nvidia.com/en-us/data-center/virtual-solutions/ https://www.nvidia.com/en-us/data-center/virtual-solutions/ TL;DR: seems to be something useful for deploying GPUs in the cloud, but I may not have understood fully.

lovedswain 6y ago
It instantiates multiple logical PCI adaptors for a single physical adaptor. The logical adaptors can then be mapped into VMs which can directly program a hardware-virtualized view of the graphics card. Intel has the same feature in their graphics and networking chips
- ur-whale 6y ago
  Thanks for the explanation, but that's more of a "this is how it works" than a "this is why it's useful". What would be the main use case?
  - wmf 6y ago
    The use case is allowing the host system and VM(s) to access the same GPU at the same time.
    - ur-whale 6y ago
      Yeah, I got that from the technical explanation. What's the practical use case, as in, when would I need this? [EDIT]: To maybe ask a better way: will this practically help me train my DNN faster? Or if I'm a cloud vendor, will this allow me to deploy cheaper GPU for my users? I guess I'm asking about the economic value of the hack.
      - lovedswain 6y ago
        Running certain ML models in VMs Running CUDA in VMs Running transcoders in VMs Running <anything that needs a GPU> in VMs

sudosysgen 6y ago

Amazing! Simply amazing! This not only enables the use of GPGPU on VMs, but also enables the use of a single GPU to virtualize Windows video games from Linux! This means that one of the major problems with Linux on the desktop for power users goes away, and it also means that we can now deploy Linux only GPU tech such as HIP on any operating system that supports this trick!

ur-whale 6y ago
> Amazing! Simply amazing! If it's such a cool feature, why does NVidia lock it away non-Tesla H/W? [EDIT]: Funny, but the answers to this question actually provide way better answers to the other question I posted in this thread (as in: what is this for).
- matheusmoreira 6y ago
  To make people pay more.
  - deleted 6y ago
    [deleted]
- sudosysgen 6y ago
  Because otherwise, people would be able to use non-Tesla GPUs for cloud compute workloads, drastically reducing the cost of cloud GPU compute, and it would also enable the use of non-Tesla GPUs as local GPGPU clusters - additionally reducing workstation GPU sales due to more efficient resource use. GPUs are a duopoly due to intellectual property laws and high costs of entry (the only companies I know of that are willing to compute are Chinese and only a result of sanctions), so for NVidia this just allows for more profit.
  - userbinator 6y ago
    Interestingly, Intel is probably the most open with its GPUs, although it wasn't always that way; perhaps they realised they couldn't compete on performance alone.

liuliu 6y ago

One thing I want to figure out (because I don't have a dedicated Windows gaming desktop), and the documentation on the internet seems sparse: it is my understanding that if I want to use PCIe passthrough with Windows VM, these GPUs cannot be available to the host machine at all, or technically it can, but I need to do some scripting to make sure the NVIDIA driver doesn't own these PCIe lanes before open Windows VM and re-enable it after shutdown? If I go with vGPU solution, I don't need to turn on / off NVIDIA driver for these PCIe lanes when running Windows VM? (I won't use these GPUs on host machine for display).

matheusmoreira 6y ago
Exactly. With GPU virtualization the driver is able to share the GPU resources with multiple systems such as the host operating system and guest virtual machine. Shame on nvidia for arbitrarily locking us out of this feature.
- judge2020 6y ago
  Is there info on this sort of usage? I'd love to use the host for NVENC and a VM guest for traditional GPU stuff, but haven't been able to find anything on doing that.
- liuliu 6y ago
  Got some time to try this now. It worked as expected, I have vgpu_vfio. However, it doesn't perfectly fit my needs. Particularly, my host system is "heavy", I need it to run CUDA etc, while the VM just to run games. However, it seems the 460.32.04 driver on host doesn't have full functionality, hence, cannot run CUDA on the host any more.
deleted 6y ago
[deleted]
Youden 6y ago
> One thing I want to figure out (because I don't have a dedicated Windows gaming desktop), and the documentation on the internet seems sparse: it is my understanding that if I want to use PCIe passthrough with Windows VM, these GPUs cannot be available to the host machine at all, or technically it can, but I need to do some scripting to make sure the NVIDIA driver doesn't own these PCIe lanes before open Windows VM and re-enable it after shutdown? The latter statement is correct. The GPU can be attached to the host but it has to be detached from the host before the VM starts using it. You may also need to get a dump of the GPU ROM and configure your VM to load it at start up. Regarding the script, mine resembles [0]. You need to remove the NVIDIA drivers and then attach the card to VFIO. And then the opposite afterwards. You may also need to image your GPU ROM [1] [0]: https://techblog.jeppson.org/2019/10/primary-vga-passthrough-in-proxmox/ https://techblog.jeppson.org/2019/10/primary-vga-passthrough... [1]: https://clayfreeman.github.io/gpu-passthrough/#imaging-the-gpu-rom https://clayfreeman.github.io/gpu-passthrough/#imaging-the-g...

neatze 6y ago

To me this is laughably naive question, but I ask it any way. My understanding is that CPU/GPU per application can make only single draw call in sequential manner. (eg. CPU->GPU->CPU->GPU) Could vgpu's be used for concurrent draw calls from multiple processes of an single application ?

milkey_mouse 6y ago
> My understanding is that CPU/GPU per application can make only single draw call in sequential manner. The limitation you're probably thinking of is in the OpenGL drivers/API, not in the GPU driver itself. OpenGL has global (per-application) state that needs to be tracked, so outside of a few special cases like texture uploading you have to only issue OpenGL calls from one thread. If applications use the lower-level Vulkan API, they can use a separate "command queue" for each thread. Both of those are graphics APIs, I'm less familiar with the compute-focused ones but I'm sure they can also process calls from multiple threads.
- milkey_mouse 6y ago
  And VGPUS are isolated from one another, that's the whole point-so using multiple in one application would be very difficult, as I don't think they can share data/memory in any way.
- neatze 6y ago
  My primitive thoughts: Threaded Computation on CPU -> Single GPU Call -> Parallel Computation on GPU -> Threaded Computation on CPU ... I wonder if it can be used in such way: Asyc Concurrent Computation on CPU -> Asyc Concurrent GPU Calls -> Parallel Time Independent Computations on GPU -> Asyc Concurrent Computation on CPU
deleted 6y ago
[deleted]

shmerl 6y ago

Is this for SR-IOV? It's too bad SR-IOV isn't supported on regular desktop AMD GPUs for example in the Linux driver.

Nullabillity 6y ago
Yes, this is basically NVidia's SR-IOV.

noodlesUK 6y ago

I wish Nvidia would open this up properly. The fact that intel integrated gpus can do GVT-G and I literally can’t buy a laptop which will do vgpu passthrough with an Nvidia card for any amount of money is infuriating.

my123 6y ago
GVT-g is gone on 10th-gen GPUs (Ice Lake) and later. Not supported on Intel dGPUs either.
- cromka 6y ago
  Oh wow. Had no idea and I was explicitly planning my next server to be Intel because of the GVT-g. Why did they abandon it?
  - deleted 6y ago
    [deleted]
  - effie 6y ago
    More like did not prioritize it. https://github.com/intel/gvt-linux/issues/126 https://github.com/intel/gvt-linux/issues/126

WrtCdEvrydy 6y ago

This is a dumb question, but which hypervisor configuration is this targeted towards to. There's a lot of detail on the link which I appreciate but maybe I missed it.

effie 6y ago
KVM, maybe also Xen.

airocker 6y ago

This is super! What would it take to abstract it similar to CPU/Memory by specifying limits only in croups? Limits could be like GPU Memory size/amount of parallelization?

minimalist 6y ago

Related but different: - nvidia-patch [0] "This patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs." - About a week ago "NVIDIA Now Allows GeForce GPU Pass-Through For Windows VMs On Linux" [1]. Note, this is only for the driver on Windows VM guests not GNU/Linux guests. Hopefully the project in the OP will mean that GPU access is finally possible on GNU/Linux guests on Xen, thank you for sharing OP. [0]: https://github.com/keylase/nvidia-patch https://github.com/keylase/nvidia-patch [1]: https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-GeForce-Virt-On-Linux https://www.phoronix.com/scan.php?page=news_item&px=NVIDIA-G...

madjam002 6y ago

I built an "X gamers/workstations 1 CPU" type-build last year and this has been the main problem, I have two GPUs, one of which is super old and I have to choose which one I want to use when I boot up a VM. Will definitely be checking this out!

DCKing 6y ago

Dual booting is for chumps. If I could run a base Linux system and arbitrarily run fully hardware accelerated VMs of multiple Linux distros, BSDs and Windows, I'd be all over that. I could pretend here that I really need the ability to quickly switch between OSes, that I'd like VM-based snapshots, or that I have big use cases to multiplex the hardware power in my desktop box like that. I really don't need it. I just want it. I really hope Intel sees this as an opportunity for their DG2 graphics cards due out later this year. If anyone from Intel is reading this: if you guys want to carve out a niche for yourself, and have power users advocate for your hardware - this is it. Enable SR-IOV for your upcoming Xe DG2 GPU line just as you do for your Xe integrated graphics. Just observe the lengths that people go to for their Nvidia cards, injecting code into their proprietary drivers just to run this. You can make this a champion feature just by not disabling something your hardware can already do. Add some driver support for it in the mix and you'll have an instant enthusiast fanbase for years to come.

strstr 6y ago
Passthrough is workable right now. It’s a pain to get set up, but it is workable. You don’t need vgpu to get the job done. I’ve had two set ups over time: one based on a jank old secondary gpu that is used by the vm host, another based on just using the jank integrated graphics on my chip. Even still, I dual boot because it just works. It always works, and boot times are crazy low for Windows these days. No fighting with drivers. No fighting with latency issues for non-passthrough devices. It all just works.
- DCKing 6y ago
  Oh I'm aware of passthrough. It's just a complete second class citizen because it isn't really virtualization, it's a hack. Virtualization is about multiplexing hardware. Passthrough is the opposite of multiplexing hardware: it's about yanking a peripheral from your host system and shoving it into one single guest VM. The fact that this yanking is poorly supported and has poor UX makes complete sense. I consider true peripheral multiplexing with true GPU virtualization to be the way of the future. It's true virtualization and doesn't even require you to sacrifice and/or babysit a single PCIe connected GPU. Passthrough is just a temporary hacky workaround that people have to apply now because there's nothing better. In the best case scenario - with hardware SR-IOV support plus basic driver support for it, enabling GPU access in your VM with SR-IOV would be a simple checkbox in the virtualization software of the host. GPU passthrough can't ever get there in terms of usability.

jarym 6y ago

Hacking at its finest! Nice

cercatrova 6y ago

For virtualized Windows from Linux, check out Looking Glass which I posted about previously https://news.ycombinator.com/item?id=22907306 https://news.ycombinator.com/item?id=22907306

albertzeyer 6y ago

The Python script actually mostly uses Frida (https://frida.re/ https://frida.re/) scripting. I haven't seen Frida before, but this looks very powerful. I did some similar (but very basic) things with GDB/LLDB scripting before but Frida seems to be done for exactly things like this.

ternox99 6y ago

I don't understand how and where I can download nvidia grid vgpu driver. Anyone can help me?

liuliu 6y ago
Register for evaluation: https://nvid.nvidia.com/dashboard/#/dashboard https://nvid.nvidia.com/dashboard/#/dashboard
- ternox99 6y ago
  Thanks :)

williesleg 6y ago

Why can't these assholes post a list of compatible gpu's, why do I have to install this shit to see?

schaefer 6y ago

There's a lot of customer loyalty on the table waiting for the first GPU manufacturer to unlock this feature on consumer grade cards without forcing us to resort to hacks.

deleted 6y ago
[deleted]
judge2020 6y ago
Not many people are foregoing a GPU given this limitation, though, except for maybe miners (which will virtualize and pirate/never activate Windows if they really need Windows).
- throwaway2048 6y ago
  that equilibrium only holds as long as your competitor doesn't offer it.

archi42 6y ago

This puts me in a tough spot: There are good reasons to go with nVidia (once they release GPUs with proper memory configuration), DLSS, RTX and now this. On the other hand, do I really want to give money to the company that locked this feature in the first place? Difficult call, but at least the ridiculous prices mean I can still think about it for a few more months.

SXX 6y ago
I'm very much AMD fanboy and I find RTX quite useless since it's only implemented in few games and work well in 2K only on really high-end cards. Yet AMD is no different there. They also locked SR-IOV for premium datacenter hardware only and they certainly want to keep some features away from consumers.
- chaz6 6y ago
  Not particularly an nvidia fanboy, but RTX can also be used to speed up rendering in Blender with the E-Cycles add-on. https://blendermarket.com/products/e-cycles https://blendermarket.com/products/e-cycles

solnyshok 6y ago

what are those "some Geforce and Quadro GPUs that share the same physical chip as the Tesla GPUs"?

broodbucket 6y ago
it's in the source https://github.com/DualCoder/vgpu_unlock/blob/0675b563acdae8f6c5eb1a760fc0e5b853b1a63e/vgpu_unlock_hooks.c#L704 https://github.com/DualCoder/vgpu_unlock/blob/0675b563acdae8...
- solnyshok 6y ago
  thanks! it is interesting. gtx 1060 3gb is ok, but rtx 3070 isn't. and it is time to upgrade from my trusty 970

Sebb767 6y ago

If anyone needs a list of currently supported CPUs, you can find it in the source code: https://github.com/DualCoder/vgpu_unlock/blob/master/vgpu_unlock#L97-L139 https://github.com/DualCoder/vgpu_unlock/blob/master/vgpu_un... (the comments behind the device ids)

opan 6y ago

Is this only for Nvidia GPUs? If so, why not put it in the title?

kkielhofner 6y ago

At the risk of piling on without value: "Amazing, simply amazing". I've been (more or less) accused of being an Nvidia fanboy on HN previously but this is an area where I've always thought Nvidia has their market segmentation wrong. Just wrong. This is great work. (Period, as in "end of sentence, mic drop").

effie 6y ago

Wow, does this mean NVIDIA consumer cards can now do multi-VM multiplexing of single GPU? On AMD side, only special cards like FirePro S7150 can do this. Does this work also on Xen? NVIDIA drivers were always non-functional with Xen+consumer cards.

effie 6y ago

The number of technology layers one must understand and control to make GPU draw what VM wants is frankly insane. Hypervisor channels, GPU driver code, graphics server API, graphics toolkit libraries - all these have several variants. It seems close to impossible to get anything done in this space. I just want to send drawing commands from a node over network to another node with a GPU. Like "draw a black rectangle 20,20,200,200 on main GPU on VM at 192.168.1.102". How would I do that in the simplest possible way? Is there some network graphics command protocol? Like X11, but simpler and faster, just the raw drawing commands.

pasnew 6y ago

Great work ! Does it limited by Nvidia's 90 day vGPU software evaluation ?

chern12 6y ago

Does this mean we'll now see miners make 3-5 vVGPUs from one 3090 for triple the hashrate per card? Because I'm already having trouble getting a 3090 as it stands.

rektide 6y ago

amd hasn't even been advertising their old "mxgpu technology" nor any follow up, & I am quite sad about it. hacks like nucleus coop[1] to run multiple instances of games on a desktop work via a shared screen & hacks, but I'd rather have a multi-head desktop with a bunch of gaming vms. [1] https://github.com/lucasassislar/nucleuscoop https://github.com/lucasassislar/nucleuscoop