3 ms·
Last I checked MIG was the only one that made hard promises about especially memory bandwidth; as long as your memory access patterns aren't secret and you have
by namibj 10mo ago
Last I checked MIG was the only one that made hard promises about especially memory bandwidth; as long as your memory access patterns aren't secret and you have enough trust in the other guests not being highly unfriendly with their cache usage behavior, you should be able to get away with much less strict isolation. Think docker vs. VMs-with-dedicated-cores.
But I thought MIG did do the job of chopping a GPU that's too big for most individual users into something that behaves very close to a literal array of smaller GPUs stuffed into the same PCIe card form factor?
Think how a Tesla K80 was pretty much just two GK210 "GPUs" on a PLX "PCIe switch" which connects them to each other and to the host. Obviously trivial to give one to each of two VMs (at least if the PLX didn't interfere with IOMMU separation or such.... for mere performance isolation it certainly sufficed (once you block a heavy user from power budget throttling the sibling, at least).
- tptacek 10mo agoCan you pass a MIG device into a KVM VM? The team we worked with didn't believe it was possible (they suggested we switch to VMWare); the MIG system interface gives you a UUID, not a PCI BDF.
- moondev 10mo agoKubevirt has some examples passing a vpgu into kvm https://kubevirt.io/user-guide/compute/host-devices/ https://kubevirt.io/user-guide/compute/host-devices/
- tptacek 10mo agoRight, vGPUs are explicitly set up to generate BDF addresses that can be passed through (but require host driver support; they're essentially paravirtualized). I'm asking about MIG.
- namibj 10mo agohttps://docs.nvidia.com/datacenter/tesla/mig-user-guide/supported-configurations.html https://docs.nvidia.com/datacenter/tesla/mig-user-guide/supp... says GPU passhtrough is supported on MIG...
- my123 10mo agoThere's a MIG vGPU mode usable for this
- tptacek 10mo agoHave you used it? How does it work? How do you drive it? We tried a lot of different things. Is it not paravirtualized, the way vGPUs are?
- my123 10mo agoIt works with SR-IOV instead of mdev afaik Still needs some host SW to drive it but actually does static partitioning IIRC it's usable through using the MIG-marked vGPU types