4 ms·

Wouldn't it be nice if the OS always allocated buffers in an address space that is guaranteed to be accessible by the peripheral device? Wouldn't it be nice if

Wouldn't it be nice if the OS always allocated buffers in an address space that is guaranteed to be accessible by the peripheral device? Wouldn't it be nice if the OS managed an IOMMU so the peripheral device could not access memory it is not allowed to access? I'm scratching my head wondering why these problems still exist today..

bpye 5y ago
> Wouldn't it be nice if the OS managed an IOMMU so the peripheral device could not access memory it is not allowed to access? I would be entirely unsurprised if a lot of device drivers didn't use the correct method to get a DMA buffer - meaning that this would just break them...
- yjftsjthsd-h 5y ago
  Okay, but Linux is a monorepo; it might be a lot of work if they can't be automatically fixed, but you could change the standard and switch over every single driver at once. (Ignoring, of course, out of tree drivers, which I suppose shouldn't be totally ignored but which aren't generally viewed as blockers.)
- temac 5y ago
  If I remember correctly, if an IOMMU is present it is enabled by default on modern Linux versions. Most drivers should be fine now.
londons_explore 5y ago
There is plenty of hardware where an OS cannot make either of those guarantees. So code needs to be written to handle those cases. And if there is code to handle the cases, it needs to be correct, and if it is correct, there is no harm in using it in other edge cases where it doesn't have a big performance hit.
- brooksbp 5y ago
  Right, so this is about supporting legacy hardware. Nobody today is designing a chip with IP that cannot support an AxADDR width that is equal to the architecture's bit width (64 or 32), right? Right? shakes fist in rage
striking 5y ago
IOMMU handling is pretty broken on a lot of motherboards, iirc. I recall having to use a patch to force Linux to ignore incorrect IOMMU group handling to use VFIO.
- strstr 5y ago
  The motherboard wasn’t broken per se, you just were trying to split an iommu group apart, which is insecure (and thus unavailable by default).
  - drbawb 5y ago
    >which is insecure Splitting IOMMU groups is not inherently insecure; doing so without ACS is insecure. It is true that's not the motherboard vendors fault - it is because Intel designed it that way. ACS (the part of PCIe which makes sane IOMMU group splitting possible) works great on my Zen products (Ryzen, Threadripper, EPYC, etc.) It also works fine on my Intel Xeon products. Intel client silicon is the odd man out here, and if you ask me that's hardly an accident. It falls neatly in line w/ their other artificial market segmentation. (They do the exact same thing w/ ECC RAM.) I'm mostly just jaded because my use-case (PCIe passthrough for a 'gaming VM') is hardly conducive to enterprise grade hardware (Xeon SKUs w/ low base clocks, headless GPUs designed for compute, etc.) yet vendors like NVidia and Intel choose to make this use-case difficult/impossible to implement on their consumer grade offerings. It gets old after a while. I, for one, am really glad AMD shook things up in this regard in the CPU space.
    - strstr 5y ago
      PCI-passthrough support for gaming is pretty niche (I say this having set it up myself), and I can understand vendors not wanting to support it. Those support tickets would be a pain. Gaming VMs are likely going to die anyway, at least for competitive gaming. Virtualization is just too powerful a tool for cheating. Many anti-cheats already ban it.
      - whitexn--g28h 5y ago
fsflover 5y ago
Here you go: https://qubes-os.org https://qubes-os.org.

bpye 5y ago

> Wouldn't it be nice if the OS managed an IOMMU so the peripheral device could not access memory it is not allowed to access? I would be entirely unsurprised if a lot of device drivers didn't use the correct method to get a DMA buffer - meaning that this would just break them...

yjftsjthsd-h 5y ago
Okay, but Linux is a monorepo; it might be a lot of work if they can't be automatically fixed, but you could change the standard and switch over every single driver at once. (Ignoring, of course, out of tree drivers, which I suppose shouldn't be totally ignored but which aren't generally viewed as blockers.)
temac 5y ago
If I remember correctly, if an IOMMU is present it is enabled by default on modern Linux versions. Most drivers should be fine now.

londons_explore 5y ago

There is plenty of hardware where an OS cannot make either of those guarantees. So code needs to be written to handle those cases. And if there is code to handle the cases, it needs to be correct, and if it is correct, there is no harm in using it in other edge cases where it doesn't have a big performance hit.

brooksbp 5y ago
Right, so this is about supporting legacy hardware. Nobody today is designing a chip with IP that cannot support an AxADDR width that is equal to the architecture's bit width (64 or 32), right? Right? shakes fist in rage

striking 5y ago

IOMMU handling is pretty broken on a lot of motherboards, iirc. I recall having to use a patch to force Linux to ignore incorrect IOMMU group handling to use VFIO.

strstr 5y ago
The motherboard wasn’t broken per se, you just were trying to split an iommu group apart, which is insecure (and thus unavailable by default).
- drbawb 5y ago
  >which is insecure Splitting IOMMU groups is not inherently insecure; doing so without ACS is insecure. It is true that's not the motherboard vendors fault - it is because Intel designed it that way. ACS (the part of PCIe which makes sane IOMMU group splitting possible) works great on my Zen products (Ryzen, Threadripper, EPYC, etc.) It also works fine on my Intel Xeon products. Intel client silicon is the odd man out here, and if you ask me that's hardly an accident. It falls neatly in line w/ their other artificial market segmentation. (They do the exact same thing w/ ECC RAM.) I'm mostly just jaded because my use-case (PCIe passthrough for a 'gaming VM') is hardly conducive to enterprise grade hardware (Xeon SKUs w/ low base clocks, headless GPUs designed for compute, etc.) yet vendors like NVidia and Intel choose to make this use-case difficult/impossible to implement on their consumer grade offerings. It gets old after a while. I, for one, am really glad AMD shook things up in this regard in the CPU space.
  - strstr 5y ago
    PCI-passthrough support for gaming is pretty niche (I say this having set it up myself), and I can understand vendors not wanting to support it. Those support tickets would be a pain. Gaming VMs are likely going to die anyway, at least for competitive gaming. Virtualization is just too powerful a tool for cheating. Many anti-cheats already ban it.
    - whitexn--g28h 5y ago

fsflover 5y ago

Here you go: https://qubes-os.org https://qubes-os.org.