5 ms·

This is a better description of what it's doing: SR-IOV: https://cdrdv2-public.intel.com/321211/pci-sig-sr-iov-primer-sr-iov-technology-paper.pdf https://cdrdv

This is a better description of what it's doing: SR-IOV: https://cdrdv2-public.intel.com/321211/pci-sig-sr-iov-primer-sr-iov-technology-paper.pdf https://cdrdv2-public.intel.com/321211/pci-sig-sr-iov-primer... S-IOV: https://cdrdv2-public.intel.com/671403/intel-scalable-io-virtualization-technical-specification.pdf https://cdrdv2-public.intel.com/671403/intel-scalable-io-vir... What they are doing is "technically" giving direct bus access, however the bus access they are giving is restricted such that the VM's accesses are all tagged and if they access anything outside the bounds they are permitted (as defined by access controls on the hardware during configuration), then you get a fault instead of the VM successfully touching anything. This is similar to how VT-d and other CPU virt extensions allow direct access to RAM but with permissioning and access control through the IOMMU. And then the other major component of SR-IOV and S-IOV is that they virtualise the interface on the PCI-E hardware itself (called virtual functions) and all of the context associated, the registers, the BAR, etc. This is akin to how VT-x and similar instructions virtualise the CPU (and registers, etc). And notably these virtual functions can be restricted via access controls, quotas, etc in hardware. So your existing VT-x extension virtualises the CPU, your existing VT-d extension virtualises the IOMMU and RAM, your existing VT-c virtualises network interfaces (but not PCI-E in general). Now SR-IOV and S-IOV virtualise the PCI-E bus w/ access control over the lanes. And now SR-IOV and S-IOV virtualise the PCI-E device hardware and their functions/interface on the bus (akin to VT-x and VT-d). Now notably S-IOV should be seen as a "SR-IOV 2.0" rather than an accompanying feature. It essentially moves the virtual function to physical function translation from the CPU or hardware in the chipset directly into the PCI-E device itself.

AshamedCaptain 1y ago
I do not understand what results in this confusion. > What they are doing is "technically" giving direct bus access, however the bus access they are giving is restricted such that the VM's accesses are all tagged This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware. > and if they access anything outside the bounds they are permitted (as defined by access controls on the hardware during configuration), then you get a fault instead of the VM successfully touching anything. Again, this is exactly what I said: you are now at the mercy of the hardware manufacturer whether there is any partitioning whatsoever. To think otherwise is wishful thinking that I do not know where it comes from. This is entirely the definition of giving the VM direct access to the hardware. There is no software-controlled emulation whatsoever going on, so you explicitly lose containment and increase your attack surface. For everything except the simplest of ethernet cards, your hardware is likely implementing this multiplexing in closed source firmware done by hardware engineers. Very likely the worst type of code ever written security-wise. > This is similar to how VT-d and other CPU virt extensions allow direct access to RAM but with permissioning and access control through the IOMMU. Not at all. Usually IOMMU is for constraining hardware that already has direct access to the RAM in the first place. > And then the other major component of SR-IOV and S-IOV is that they virtualise the interface on the PCI-E hardware itself (called virtual functions) Is this the source of the confusion? That because it is called virtual you think this virtualized somehow? It is the reason I call it partition because it is much closer to what it is (from a hw point of view). > your existing VT-x extension virtualises the CPU, your existing VT-d extension virtualises the IOMMU and RAM, your existing VT-c virtualises network interfaces (but not PCI-E in general This is meaningless because it mixes and matches everything. What does it mean to "virtualize the RAM"? RAM is already virtualized by the normal MMU, no VT-d needed at all. Hardware is the one who may require to also have its RAM access virtualized so its idea of memory matches that of the VM directly accessing hardware (instead of through a software emulation layer), and that is what benefits from an IOMMU (but does not generally require it, see GART and VT-c). But the entire point of this is again to give the VM direct access to hardware! What is it exactly that you want to refute from this?
- OneDeuxTriSeiGo 1y ago
  > This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware. Yes but the whole point is that it's moving the isolation of the VM's access from software to hardware. Yes you are giving direct access to a subset of hardware but that subset of hardware is configured from outside the VM's access to restrict the VM's access. > Again, this is exactly what I said: you are now at the mercy of the hardware manufacturer whether there is any partitioning whatsoever. To think otherwise is wishful thinking that I do not know where it comes from. That's not actually true to my knowledge. S-IOV and SR-IOV require hardware support. Sure the manufacturer can do a shit job at implementing it but both S-IOV and SR-IOV require partitioning. But if you are granting your VMs S-IOV or SR-IOV access to hardware, you are at minimum implicitly trusting that the hardware manufacturer implemented the spec correctly. > There is no software-controlled emulation whatsoever going on, so you explicitly lose containment and increase your attack surface. This is true but the same is true of VT-x, VT-d, etc (i.e. the commonplace virtualisation extensions). It is no less true with S-IOV or SR-IOV other than by them being newer and less "battletested". If you use virtualisation extensions you are no longer doing pure software virtualisation anyways. > For everything except the simplest of ethernet cards, your hardware is likely implementing this multiplexing in closed source firmware done by hardware engineers. Very likely the worst type of code ever written security-wise. The exact same applies to the microcode and internal firmware on modern CPUs and the associated chipset. > Not at all. Usually IOMMU is for constraining hardware that already had direct access to the RAM in the first place. Yes. And VT-d extends this for VMs by introducing hardware level IO, interrupt, and DMA remapping so that the host doesn't need to do software level remapping instead. > Is this the source of the confusion? That because it is called virtual you think this virtualized somehow? It is the reason I call it partition because it is much closer to what it is (from a hw point of view). I call it virtualisation because it is virtualisation. In SR-IOV it is still virtualisation but yes it is architecturally similar to partitioning with access controls however that is still virtualisation, it just prevents nesting. With S-IOV however it is full on-hardware virtualisation and supports nesting virtual devices. > What does it mean to "virtualize the RAM"? RAM is already virtualized by the normal MMU, no VT-d needed at all. Hardware is the one who may require to also have its RAM access virtualized so its idea of memory matches that of the VM directly accessing hardware (instead of through a software emulation layer), and that is what benefits from an IOMMU (but does not generally require it, see GART and VT-c). Yes I was playing loose with the terminology. Yes RAM is already virtualised (to a certain degree) but VT-d extends that completely and allows arbitrary nesting. And yes VT-d is not required for virtualisation but it is important in accelerating virtualisation by moving it from software virt to hardware virt. > But the entire point of this is again to give the VM direct access to hardware! What is exactly that you want to refute from this? I think the disconnect here is that I (and I assume others) are operating under the assumption that giving the VM access to an access controlled and permissioned subset of the hardware through hardware virtualisation extensions/frameworks wouldn't fall under "giving the VM direct access to the hardware" any more than CPU virtualisation extensions do (which are essentially always enabled). ---------- Edit: Oh I should also add in that another commenter was in our comment chain. I just realised they were the one arguing that SR-IOV/S-IOV wouldn't make you at the mercy of the HW manufacturer to implement the isolation and virtualisation functionality correctly. That may help clear up some misunderstanding because I 100% get that you are reliant on the HW manufacturer implementing the feature correctly for it to be secure.

AshamedCaptain 1y ago

I do not understand what results in this confusion. > What they are doing is "technically" giving direct bus access, however the bus access they are giving is restricted such that the VM's accesses are all tagged This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware. > and if they access anything outside the bounds they are permitted (as defined by access controls on the hardware during configuration), then you get a fault instead of the VM successfully touching anything. Again, this is exactly what I said: you are now at the mercy of the hardware manufacturer whether there is any partitioning whatsoever. To think otherwise is wishful thinking that I do not know where it comes from. This is entirely the definition of giving the VM direct access to the hardware. There is no software-controlled emulation whatsoever going on, so you explicitly lose containment and increase your attack surface. For everything except the simplest of ethernet cards, your hardware is likely implementing this multiplexing in closed source firmware done by hardware engineers. Very likely the worst type of code ever written security-wise. > This is similar to how VT-d and other CPU virt extensions allow direct access to RAM but with permissioning and access control through the IOMMU. Not at all. Usually IOMMU is for constraining hardware that already has direct access to the RAM in the first place. > And then the other major component of SR-IOV and S-IOV is that they virtualise the interface on the PCI-E hardware itself (called virtual functions) Is this the source of the confusion? That because it is called virtual you think this virtualized somehow? It is the reason I call it partition because it is much closer to what it is (from a hw point of view). > your existing VT-x extension virtualises the CPU, your existing VT-d extension virtualises the IOMMU and RAM, your existing VT-c virtualises network interfaces (but not PCI-E in general This is meaningless because it mixes and matches everything. What does it mean to "virtualize the RAM"? RAM is already virtualized by the normal MMU, no VT-d needed at all. Hardware is the one who may require to also have its RAM access virtualized so its idea of memory matches that of the VM directly accessing hardware (instead of through a software emulation layer), and that is what benefits from an IOMMU (but does not generally require it, see GART and VT-c). But the entire point of this is again to give the VM direct access to hardware! What is it exactly that you want to refute from this?

OneDeuxTriSeiGo 1y ago
> This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware. Yes but the whole point is that it's moving the isolation of the VM's access from software to hardware. Yes you are giving direct access to a subset of hardware but that subset of hardware is configured from outside the VM's access to restrict the VM's access. > Again, this is exactly what I said: you are now at the mercy of the hardware manufacturer whether there is any partitioning whatsoever. To think otherwise is wishful thinking that I do not know where it comes from. That's not actually true to my knowledge. S-IOV and SR-IOV require hardware support. Sure the manufacturer can do a shit job at implementing it but both S-IOV and SR-IOV require partitioning. But if you are granting your VMs S-IOV or SR-IOV access to hardware, you are at minimum implicitly trusting that the hardware manufacturer implemented the spec correctly. > There is no software-controlled emulation whatsoever going on, so you explicitly lose containment and increase your attack surface. This is true but the same is true of VT-x, VT-d, etc (i.e. the commonplace virtualisation extensions). It is no less true with S-IOV or SR-IOV other than by them being newer and less "battletested". If you use virtualisation extensions you are no longer doing pure software virtualisation anyways. > For everything except the simplest of ethernet cards, your hardware is likely implementing this multiplexing in closed source firmware done by hardware engineers. Very likely the worst type of code ever written security-wise. The exact same applies to the microcode and internal firmware on modern CPUs and the associated chipset. > Not at all. Usually IOMMU is for constraining hardware that already had direct access to the RAM in the first place. Yes. And VT-d extends this for VMs by introducing hardware level IO, interrupt, and DMA remapping so that the host doesn't need to do software level remapping instead. > Is this the source of the confusion? That because it is called virtual you think this virtualized somehow? It is the reason I call it partition because it is much closer to what it is (from a hw point of view). I call it virtualisation because it is virtualisation. In SR-IOV it is still virtualisation but yes it is architecturally similar to partitioning with access controls however that is still virtualisation, it just prevents nesting. With S-IOV however it is full on-hardware virtualisation and supports nesting virtual devices. > What does it mean to "virtualize the RAM"? RAM is already virtualized by the normal MMU, no VT-d needed at all. Hardware is the one who may require to also have its RAM access virtualized so its idea of memory matches that of the VM directly accessing hardware (instead of through a software emulation layer), and that is what benefits from an IOMMU (but does not generally require it, see GART and VT-c). Yes I was playing loose with the terminology. Yes RAM is already virtualised (to a certain degree) but VT-d extends that completely and allows arbitrary nesting. And yes VT-d is not required for virtualisation but it is important in accelerating virtualisation by moving it from software virt to hardware virt. > But the entire point of this is again to give the VM direct access to hardware! What is exactly that you want to refute from this? I think the disconnect here is that I (and I assume others) are operating under the assumption that giving the VM access to an access controlled and permissioned subset of the hardware through hardware virtualisation extensions/frameworks wouldn't fall under "giving the VM direct access to the hardware" any more than CPU virtualisation extensions do (which are essentially always enabled). ---------- Edit: Oh I should also add in that another commenter was in our comment chain. I just realised they were the one arguing that SR-IOV/S-IOV wouldn't make you at the mercy of the HW manufacturer to implement the isolation and virtualisation functionality correctly. That may help clear up some misunderstanding because I 100% get that you are reliant on the HW manufacturer implementing the feature correctly for it to be secure.