2 ms·
SR-IOV doesn't make sense on most GPUs, because they have their own MMUs already with different semantics. SR-IOV is terrible for dealing with a large bank of
by monocasa 6y ago
SR-IOV doesn't make sense on most GPUs, because they have their own MMUs already with different semantics. SR-IOV is terrible for dealing with a large bank of RAM on the target device, because the MMU is in the wrong part of the architecture. It's all the way on the root complex, so all GPU VRAM reads and writes would need to take the slow path through PCI-E and back. This is basically a nonstarter for GPUs that have their own VRAM. They also don't cover the GPU specific caching information that's normally stored in the GPU page tables either.
Intel is able to get away with it for integrated GPUs because they can codesign their system level IO-MMU with the GPU MMU and it doesn't have it's own VRAM bank. I'd bet dollars to donuts that Intel's new Xe discrete GPU doesn't support SR-IOV even though the integrated versions of the same core will.
- wtallis 6y agoHow does AMD deal with this on their server discrete GPUs that support SR-IOV?
- monocasa 6y agoIt's not really SR-IOV in the way you'd generally consider it as it only virtualizes the GPU's access to main memory not VRAM. That still requires that you setup GPU page tables and manage them on the host side with a bridge driver in the guest like Microsoft's work here is doing. The same with Nvidia's A100.