3 ms·

> This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware.

> This is exactly what I know and what I said in my original post: a way to identify which VM is accessing what. For... giving that VM access to the hardware. Yes but the whole point is that it's moving the isolation of the VM's access from software to hardware. Yes you are giving direct access to a subset of hardware but that subset of hardware is configured from outside the VM's access to restrict the VM's access. > Again, this is exactly what I said: you are now at the mercy of the hardware manufacturer whether there is any partitioning whatsoever. To think otherwise is wishful thinking that I do not know where it comes from. That's not actually true to my knowledge. S-IOV and SR-IOV require hardware support. Sure the manufacturer can do a shit job at implementing it but both S-IOV and SR-IOV require partitioning. But if you are granting your VMs S-IOV or SR-IOV access to hardware, you are at minimum implicitly trusting that the hardware manufacturer implemented the spec correctly. > There is no software-controlled emulation whatsoever going on, so you explicitly lose containment and increase your attack surface. This is true but the same is true of VT-x, VT-d, etc (i.e. the commonplace virtualisation extensions). It is no less true with S-IOV or SR-IOV other than by them being newer and less "battletested". If you use virtualisation extensions you are no longer doing pure software virtualisation anyways. > For everything except the simplest of ethernet cards, your hardware is likely implementing this multiplexing in closed source firmware done by hardware engineers. Very likely the worst type of code ever written security-wise. The exact same applies to the microcode and internal firmware on modern CPUs and the associated chipset. > Not at all. Usually IOMMU is for constraining hardware that already had direct access to the RAM in the first place. Yes. And VT-d extends this for VMs by introducing hardware level IO, interrupt, and DMA remapping so that the host doesn't need to do software level remapping instead. > Is this the source of the confusion? That because it is called virtual you think this virtualized somehow? It is the reason I call it partition because it is much closer to what it is (from a hw point of view). I call it virtualisation because it is virtualisation. In SR-IOV it is still virtualisation but yes it is architecturally similar to partitioning with access controls however that is still virtualisation, it just prevents nesting. With S-IOV however it is full on-hardware virtualisation and supports nesting virtual devices. > What does it mean to "virtualize the RAM"? RAM is already virtualized by the normal MMU, no VT-d needed at all. Hardware is the one who may require to also have its RAM access virtualized so its idea of memory matches that of the VM directly accessing hardware (instead of through a software emulation layer), and that is what benefits from an IOMMU (but does not generally require it, see GART and VT-c). Yes I was playing loose with the terminology. Yes RAM is already virtualised (to a certain degree) but VT-d extends that completely and allows arbitrary nesting. And yes VT-d is not required for virtualisation but it is important in accelerating virtualisation by moving it from software virt to hardware virt. > But the entire point of this is again to give the VM direct access to hardware! What is exactly that you want to refute from this? I think the disconnect here is that I (and I assume others) are operating under the assumption that giving the VM access to an access controlled and permissioned subset of the hardware through hardware virtualisation extensions/frameworks wouldn't fall under "giving the VM direct access to the hardware" any more than CPU virtualisation extensions do (which are essentially always enabled). ---------- Edit: Oh I should also add in that another commenter was in our comment chain. I just realised they were the one arguing that SR-IOV/S-IOV wouldn't make you at the mercy of the HW manufacturer to implement the isolation and virtualisation functionality correctly. That may help clear up some misunderstanding because I 100% get that you are reliant on the HW manufacturer implementing the feature correctly for it to be secure.

AshamedCaptain 1y ago
> Yes but the whole point is that it's moving the isolation of the VM's access from software to hardware. Yes you are giving direct access to a subset of hardware but that subset of hardware is configured from outside the VM's access to restrict the VM's access. But who is actually gating access to this "subset" (which normally isn't a subset of functionality anyway) ? Answer: the hardware. Before, it was software who was emulating hardware and implementing whatever checks you wanted. Now, the VM OS is directly accessing the hardware, banging its registers, and you literally depend on the hardware to enforce any kind of isolation between accesses from the VMs. > This is true but the same is true of VT-x, VT-d, etc (i.e. the commonplace virtualisation extensions). It is no less true with S-IOV or SR-IOV other than by them being newer and less "battletested". ". If you use virtualisation extensions you are no longer doing pure software virtualisation anyways. No, this is not the correct analogy. Even without VT-x, CPUs since the 386 era are already designed to execute untrusted code. Adding VT-x on it changes a bit the picture but it is almost an irrelevant change in global architecture overall, since the CPU is in any case is directly executing VM guest code (see early virtualizers which did plenty well without VT-x). Here, you are allowing untrusted code direct access to hardware that has never even imagined the idea of being ever accessed by untrusted software, or even user level code to being with for most it (very few exceptions such as GPUs). The difference in the size of the security boundary is gigantic, even hard to visualize. The correct analogy would be to if you were switching from say a JavaScript VM generting native cpu code into directly executing native CPU code directly downloaded from the internet. On a 8086 level CPU with a haphazardly added MMU on top of it. Sure, works on theory. In practice, it will make everyone shiver (and with reason). That is the proper analogy. The discussion about SRIOV is a red herring because these technologies are about allowing this direct hardware access. It is not that SRIOV is a firewall between the hardware and the VM (or whatever it is that you envision). They are technologies entirely designed to facilitate this direct hardware access, not prevent or constrain it in any way.
- OneDeuxTriSeiGo 1y ago
  > Before, it was software who was emulating hardware and implementing whatever checks you wanted. Now, the VM OS is directly accessing the hardware, banging its registers, and you literally depend on the hardware to enforce any kind of isolation between accesses from the VMs. This hasn't been true for decades. CPUs have been leaving virtualisation almost entirely to the hardware. For the most part all software was doing was configuring the hardware and injecting a bit of glue here and there unless you were full on emulating another architecture. > Here, you are allowing untrusted code direct access to hardware that has never even imagined the idea of being ever accessed by untrusted software, or even user level code to being with for most it (very few exceptions such as GPUs). If the device supports SR-IOV or S-IOV then they had to engineer the product to meet the spec. It's not like this is just a switch being enabled on old hardware. Every device on the stack has to support the standard and therefore is designed to at least attempt to respect the security boundaries those specs impose. > The correct analogy would be to if you were switching from say a JavaScript VM generting native cpu code into directly executing native CPU code directly downloaded from the internet. This is exactly what every modern browser does. Chrome's V8 JS engine parses JS and generates V8 bytecode. Then at runtime V8 JIT compiles that bytecode into native machine code and executes that native code on the hardware. That's not interpreting the JS, it's actually compiling the JS into native code running on the CPU (using prediction to make sure the compilation is done before the codepaths are expected to be executed). > On a 8086 level CPU with a haphazardly added MMU on top of it. Sure, works on theory. In practice, it will make everyone shiver (and with reason). That is the proper analogy. This also isn't true. Peer to Peer DMA support has been commonplace in consumer PCI-E devices (mainly NVME, network HBAs, and GPUs) for years now and has been available in datacenter, etc for a decade at least. > On a 8086 level CPU with a haphazardly added MMU on top of it. Also minor nit but the 80286 (the 3rd gen of 8086 CPUs, released less than 4 years after the original 8086) had an integrated MMU with proper segmentation support. Additionally MMUs long predate the 8086, it just didn't initially include an integrated one because it didn't need to for the market segment it was targeting). > The discussion about SRIOV is a red herring because these technologies are about allowing this direct hardware access. It is not that SRIOV is a firewall between the hardware and the VM (or whatever it is that you envision). They are technologies entirely designed to facilitate this direct hardware access, not prevent or constrain it in any way. Again this is just not true. They provide a framework for segmentation of hardware and enforcing isolation of those segments. That is absolutely intended for "preventing and constraining" access to hardware outside of what the host configures. ------ If you can provide some citations of how SR-IOV or S-IOV doesn't do what it claims to, I'm happy to continue this conversation.

AshamedCaptain 1y ago

> Yes but the whole point is that it's moving the isolation of the VM's access from software to hardware. Yes you are giving direct access to a subset of hardware but that subset of hardware is configured from outside the VM's access to restrict the VM's access. But who is actually gating access to this "subset" (which normally isn't a subset of functionality anyway) ? Answer: the hardware. Before, it was software who was emulating hardware and implementing whatever checks you wanted. Now, the VM OS is directly accessing the hardware, banging its registers, and you literally depend on the hardware to enforce any kind of isolation between accesses from the VMs. > This is true but the same is true of VT-x, VT-d, etc (i.e. the commonplace virtualisation extensions). It is no less true with S-IOV or SR-IOV other than by them being newer and less "battletested". ". If you use virtualisation extensions you are no longer doing pure software virtualisation anyways. No, this is not the correct analogy. Even without VT-x, CPUs since the 386 era are already designed to execute untrusted code. Adding VT-x on it changes a bit the picture but it is almost an irrelevant change in global architecture overall, since the CPU is in any case is directly executing VM guest code (see early virtualizers which did plenty well without VT-x). Here, you are allowing untrusted code direct access to hardware that has never even imagined the idea of being ever accessed by untrusted software, or even user level code to being with for most it (very few exceptions such as GPUs). The difference in the size of the security boundary is gigantic, even hard to visualize. The correct analogy would be to if you were switching from say a JavaScript VM generting native cpu code into directly executing native CPU code directly downloaded from the internet. On a 8086 level CPU with a haphazardly added MMU on top of it. Sure, works on theory. In practice, it will make everyone shiver (and with reason). That is the proper analogy. The discussion about SRIOV is a red herring because these technologies are about allowing this direct hardware access. It is not that SRIOV is a firewall between the hardware and the VM (or whatever it is that you envision). They are technologies entirely designed to facilitate this direct hardware access, not prevent or constrain it in any way.

OneDeuxTriSeiGo 1y ago
> Before, it was software who was emulating hardware and implementing whatever checks you wanted. Now, the VM OS is directly accessing the hardware, banging its registers, and you literally depend on the hardware to enforce any kind of isolation between accesses from the VMs. This hasn't been true for decades. CPUs have been leaving virtualisation almost entirely to the hardware. For the most part all software was doing was configuring the hardware and injecting a bit of glue here and there unless you were full on emulating another architecture. > Here, you are allowing untrusted code direct access to hardware that has never even imagined the idea of being ever accessed by untrusted software, or even user level code to being with for most it (very few exceptions such as GPUs). If the device supports SR-IOV or S-IOV then they had to engineer the product to meet the spec. It's not like this is just a switch being enabled on old hardware. Every device on the stack has to support the standard and therefore is designed to at least attempt to respect the security boundaries those specs impose. > The correct analogy would be to if you were switching from say a JavaScript VM generting native cpu code into directly executing native CPU code directly downloaded from the internet. This is exactly what every modern browser does. Chrome's V8 JS engine parses JS and generates V8 bytecode. Then at runtime V8 JIT compiles that bytecode into native machine code and executes that native code on the hardware. That's not interpreting the JS, it's actually compiling the JS into native code running on the CPU (using prediction to make sure the compilation is done before the codepaths are expected to be executed). > On a 8086 level CPU with a haphazardly added MMU on top of it. Sure, works on theory. In practice, it will make everyone shiver (and with reason). That is the proper analogy. This also isn't true. Peer to Peer DMA support has been commonplace in consumer PCI-E devices (mainly NVME, network HBAs, and GPUs) for years now and has been available in datacenter, etc for a decade at least. > On a 8086 level CPU with a haphazardly added MMU on top of it. Also minor nit but the 80286 (the 3rd gen of 8086 CPUs, released less than 4 years after the original 8086) had an integrated MMU with proper segmentation support. Additionally MMUs long predate the 8086, it just didn't initially include an integrated one because it didn't need to for the market segment it was targeting). > The discussion about SRIOV is a red herring because these technologies are about allowing this direct hardware access. It is not that SRIOV is a firewall between the hardware and the VM (or whatever it is that you envision). They are technologies entirely designed to facilitate this direct hardware access, not prevent or constrain it in any way. Again this is just not true. They provide a framework for segmentation of hardware and enforcing isolation of those segments. That is absolutely intended for "preventing and constraining" access to hardware outside of what the host configures. ------ If you can provide some citations of how SR-IOV or S-IOV doesn't do what it claims to, I'm happy to continue this conversation.