6 ms·

The Raspberry Pi does not support PCIe I/O BAR space. That's not a hard requirement to run a modern GPU as far as I know, but drivers do kind of assume it's ava

The Raspberry Pi does not support PCIe I/O BAR space. That's not a hard requirement to run a modern GPU as far as I know, but drivers do kind of assume it's available. In particular, VBIOS stuff often needs it. The message from the Radeon driver is not a hard error, but presumably other things break as a result. The Broadcom SoCs used on Pis still suck, and although this can probably be made to work, someone familiar with this codebase needs to care enough to fix the drivers to make it happen. It's probably quite doable for the open source drivers, but I wouldn't expect Nvidia's blob to work any time soon unless they take the time to fix it. Not all PCIe root ports are equal. Some are, shall we say, less than amenable to devices more complicated than a USB controller. Looking through the discussion here, there is some rather concerning stuff about DMA mappings (e.g. the thing with 64bit accesses not being supported). It's possible the Pi4 PCIe implementation is actually too broken to support these GPUs, though I haven't read through the whole thread in detail. It certainly looks pretty messy. https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/4 https://github.com/geerlingguy/raspberry-pi-pcie-devices/iss...

geerlingguy 5y ago
Thanks for taking a glance. There's a groundswell of people commenting "get marcan to take a look". And I would ask, but you're already doing some really awesome work over on the M1 side and I've learned enough to fill up two of my brains already from that. At this point I kinda have to wave the white flag on the driver issues because I'm not deep enough on the kernel / PCI spec side to be productive figuring out the holes on the Pi's 'messy' bus :)
- monocasa 5y ago
  I spent a night looking through the radeon driver and register documentation, and it seems to fall back on sane defaults without being able to access IO space, it just prints a nasty gram. All of the expected IO space is mirrored in the MMIO space and the VBIOS code will just use that as a fallback pathway just fine. One sort of cheap way to get a sense of what's actually going on (at least way cheaper than a PCIe protocol analyzer) might be to put your hands on one of the USB3382 cards (the poor man's PCIe protocol analyzer for those who aren't that comfortable with FPGAs). Those are really neat soft PCIe devices that bridge the PCIe transactions over USB3 to another host computer and can do things like emulate other devices or send arbitrary TLPs. There's some hacker con talks about using them to attack systems by using their DMA TLPs as a read/write primitive. Using that to either bridge to a card on the host computer and tap the PCIe bus, or send your own TLPs to verify that stuff like 64 bit transactions are actually a problem and not a red herring. You can use a FPGA for the same thing pretty easily, but a USB3382 might be a lot closer to the skill set that you feel comfortable with.

geerlingguy 5y ago

Thanks for taking a glance. There's a groundswell of people commenting "get marcan to take a look". And I would ask, but you're already doing some really awesome work over on the M1 side and I've learned enough to fill up two of my brains already from that. At this point I kinda have to wave the white flag on the driver issues because I'm not deep enough on the kernel / PCI spec side to be productive figuring out the holes on the Pi's 'messy' bus :)

monocasa 5y ago
I spent a night looking through the radeon driver and register documentation, and it seems to fall back on sane defaults without being able to access IO space, it just prints a nasty gram. All of the expected IO space is mirrored in the MMIO space and the VBIOS code will just use that as a fallback pathway just fine. One sort of cheap way to get a sense of what's actually going on (at least way cheaper than a PCIe protocol analyzer) might be to put your hands on one of the USB3382 cards (the poor man's PCIe protocol analyzer for those who aren't that comfortable with FPGAs). Those are really neat soft PCIe devices that bridge the PCIe transactions over USB3 to another host computer and can do things like emulate other devices or send arbitrary TLPs. There's some hacker con talks about using them to attack systems by using their DMA TLPs as a read/write primitive. Using that to either bridge to a card on the host computer and tap the PCIe bus, or send your own TLPs to verify that stuff like 64 bit transactions are actually a problem and not a red herring. You can use a FPGA for the same thing pretty easily, but a USB3382 might be a lot closer to the skill set that you feel comfortable with.