7 ms·
Raspberry Pi PCIe Devices
- sitkack 6y agoWhere do we recommend new PCIe devices for you to test? Have you tried Mellanox Infiniband and RDMA?
- geerlingguy 6y agoCheck out the linked GitHub repo: https://github.com/geerlingguy/raspberry-pi-pcie-devices https://github.com/geerlingguy/raspberry-pi-pcie-devices — for any card I'm actually testing, I have an open issue. For proposals or discussions around a certain topic, there are some ongoing conversations in the Discussions tab. Specifically concerning Infiniband, check out this issue: https://github.com/geerlingguy/raspberry-pi-pcie-devices/discussions/12 https://github.com/geerlingguy/raspberry-pi-pcie-devices/dis... - I'd be willing to test, but currently I have a bit of a backlog of other gear I'm getting through testing :)
- Namidairo 6y agoWouldn't you saturate the bus fairly easily with a Mellanox card? I can't imagine the experience would be optimal.
- geerlingguy 6y agoFrom all my testing, it looks like the maximum throughput you can get on the entire PCIe x1 gen 2 lane on the CM4 is about 3.2-3.4 Gbps. But that's still better than the current maximum of 1 Gbps over Ethernet, or the ~3 Gbps I have tested through a USB 3.0 card.
- Uehreka 6y agoI love a good “PCI Challenge”, where someone tries to see how much functionality they can wring out of PCI devices in a weird situation. Linus Tech Tips has a bunch of good ones: - How crazy of a device can you use with a server motherboard’s PCI hot-swap BIOS feature: https://youtu.be/YigN2mkQMPc https://youtu.be/YigN2mkQMPc - How many PCI extenders can you use with a high end graphics card: https://youtu.be/q5xvwPa3r7M https://youtu.be/q5xvwPa3r7M
- marcan_42 6y agoAh, but have you ever run PCIe over 115200 baud serial? :) We did that for the PS4 hardware experiments, and that also involved a PCIe hotplug capable motherboard, as we had a PS4 southbridge patched in as a PCIe device on a normal PC, and every time we powered on the PS4 that was effectively hotplugging it. Fun times. These days PCIe hotplug is a consumer technology, it's called Thunderbolt (it's hotplug at the logical layer, not physical PCIe lanes, but to the OS/software it's about the same). PS4 talk including the PCIe story: https://youtu.be/QMiubC6LdTA https://youtu.be/QMiubC6LdTA
- Roritharr 6y agoI remember that talk fondly, thanks for all the fun!
- jchw 6y agoI read the first sentence of this post without noticing your username and thought, “wait, wasn’t that marcan?” — of course it was :) It’s a good time to rewatch that classic talk (and probably some other old ccc talks, there’s a wealth of great ones.)
- moron4hire 6y agoThis is something that I miss about forums of yore and sort of hate about HN, Reddit, and their ilk: the deprioritization of a post author's identity. I miss being able to scan through a forum's post listing and readily pick out names of posters I recognized after being involved in the site for a while. It was an additional signal of post quality (or lack thereof, depending on the user). Here, the username is hidden in a sentence of status text, on the same visual level of the number of minutes ago the post was made, all of which is rendered in a light grey font. It's clearly intentional, and I can't for the life of me figure out why the forum owners don't want users to be able to build a consistent identity.
- wrs 6y agoI think it’s always been that way on HN—it could just be arbitrary, not a deliberate choice. I have noticed around here that if you indicate you might use someone’s identity or history as a signal of quality, you’ll get a lot of “argument by authority” backlash. (Except in the cases of someone who built the thing you’re talking about, and tptacek. :) )
- vkdelta 6y agoPlease add m.2 lte and 5g modules with sim slot. They make good for IOT projects. I have seen people doing projects. Two vendors are quectel and Sierra wireless based off Qualcomm radios.
- geerlingguy 6y agoIt looks like the MCUZone CM4 4G mentioned here [1] is intended for just that purpose! I would be happy to try out a chipset provided it's not too expensive, and (ideally) it has good Linux driver support. Drivers seem to be the dealbreaker for a few cards, if they rely on PCI express features the Pi OS or SoC currently don't support (like BIOS, or sort-of MSI-X support). [1] https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/25#issuecomment-745730511 https://github.com/geerlingguy/raspberry-pi-pcie-devices/iss...
- MisterTea 6y agoThose need m.2 B key slots which are not compatible with m.2 M key NVMe disks. This is why I'm not crazy about the m.2 "standard".
- nicolaslem 6y agoTo be fair the same thing can be said about its predecessor the miniPCIe slot.
- MisterTea 6y agoTrue but that was only really for the msata version. m.2 has about half a dozen keys as of now. Like USB-C it tries to cram too many mutually incompatible electrical interfaces into one plug.
- jand 6y agoIt might be out of scope, but: Can you boot from the m.2 SSD? Do you have benchmarks for m.2 SSD? This would make my day, as all i know of is some USB-dongled SSD that (back then) could not be booted from. Sry, if this is already standard today, i am a bit out of the loop.
- geerlingguy 6y agoFollow https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/32 https://github.com/geerlingguy/raspberry-pi-pcie-devices/iss... for progress. You can't boot directly from the SSD without a microSD card, but it (or SATA support for boot) may come soon... just a question of priorities for the Pi Engineering team.
- dvdbloc 6y agoDo you happen to know if booting from the SSD is limited by the closed source bits of the Pi system or is this an effort the open source community can possibly contribute?
- geerlingguy 6y agoClosed source firmware bits, unfortunately.
- jrwr 6y agoits mostly that the bootloader has to be patched to see the devices, this is no easy feat as it would bloat the bootloader pretty hard. a chain boot of MicroSD to SSD is best at this time.
- tyingq 6y agoI imagine it's limited quite a bit by the odd way the Pi boots. It's the GPU, not the CPU (or a BIOS type setup), that initially runs the bootloader.
- tmzt 6y agoWould a chip (such as a CPLD/FPGA) implementing mmc/sd be an option? It could host a chainloader with PCIe support and serve as a kind of rom. Also, is there any work on kernel level x86 emulation on arm for expansion roms?
- 2sk21 6y agoGraphics cards presumably need an external power supply?
- geerlingguy 6y agoYes; thus the testing of the powered external 16x risers (see the last section of the cards).
- mschuster91 6y agoI'd be interested if Thunderbolt 2 or 3 cards could be made to work with the Pi.
- qwerty456127 6y agoIMHO PCIe (and SDRAM) should be the only interface exposed by a modern CPU. On-board graphics, audio and NICs usually are shitty anyway. SATA is not relevant any more - we better plug an NVMe SSD into PCIe (or get a controller if actually needed). Full USB-C (or ongoing USB-4) means Thunderbolt which is PCIe. Why waste silicon on anything but PCIe then?
- jychang 6y agoPower and convenience. Power usage by PCIe is a factor for mobile devices. And convenience, in that "shitty" onboard graphics, audio, and NICs are good enough for 90% of the use cases for most devices.
- viraptor 6y ago> On-board graphics, audio and NICs usually are shitty anyway. Onboard graphics, audio and NICs are perfectly functional and more than enough for most people. Both in work environments and at homes. They've not been in a shitty category for years now. They won't break any records, but they will display websites at high resolution, play clear music, and provide more bandwidth than the link to the internet can handle. Just look at the number of Lenovo's AIOs (and similar solitons) in offices. > Full USB-C (or ongoing USB-4) means Thunderbolt which is PCIe. USB 2 is a fraction of the price. (And power and board space)
- cubano 6y agoI really love the pushback here. Yes how awesome it is you are privileged enough to always be able to acquire the best of the best and look down on other "inferior" devices. But 99% of the rest of us just need things that work and allow us access to good performance.
- qwerty456127 6y agoI actually use everything built-in because what I use are a decade-old business laptop, an office desktop and a RaspberryPi. But going to buy a good desktop (as I used to have 2 decades ago) I would rather buy a good mainboard with as many PCIe slots as possible (and no built-in devices if this were possible), the cheapest discrete cards for everything and upgrade them as I see fit. I doubt the cheapest/used discrete card for anything is going to be worse or more expensive than a built-in non-removable version.
- flyinghamster 6y agoTelephony cards (Sangoma/Digium) would be interesting to try as well, though if you were trying to run FXS ports you would need an additional cable and PSU to supply 12V to the card.
- mikece 6y agoMy first thought was "can you boot from a PCIe SSD?" (looks like the answer is "no" from what I see). I wish there was a RPi with 16GB or 32GB of storage soldered to the board. Where I work we're using Raspberry Pi devices for industrial use and corruption of/wearing out the microSD card has been something we've wondered about. We could just buy premium microSD cards but having a "proper" SSD soldered to the board would be very helpful as well.
- tyingq 6y agoThere's the compute module...32GB of eMMC is an option. https://www.raspberrypi.org/products/compute-module-4/?variant=raspberry-pi-cm4001000 https://www.raspberrypi.org/products/compute-module-4/?varia...
- geerlingguy 6y agoAnd in some good news on that front, the eMMC storage on the CM4 is twice as fast in many cases as older generations of Pi CMs, as it uses the full 8 bit eMMC bus. It's finally faster (for random IO at least) than any microSD card I've tested.
- amenod 6y agoWe've had good success with making SD cards read-only. Of course this means that any data needs to be sent to a service for saving... YMMV.
- pmiller2 6y ago+1 to this. I've seen this done as well, and it works marvelously. SD cards last half of forever if all you're using them for is a boot device.
- dpool 6y agoIf the system can see the PCIe SSD, you could always boot off the microSD card but use the PCIe SSD as the primary storage device. Definitely worth testing imo.
- deleted 6y ago
- OJFord 6y agoIt's a shame there isn't something between the Model B and the Compute Module, like the CM but with a built-in PCIe header say. There have been a few popular submissions using multiple Pis in clusters recently, really the only reason not to use CMs there js the complexity of the (limited) IO that is needed, and the CM development board (for each one) dominates the cost and would be motr expensive than Model Bs.
- bionade24 6y agoThat's why I use RockPro64 instead, they have PCIe for a long time.
- geerlingguy 6y agoThere are a few addon boards (for $15-30, most shipping after the new year) that take the CM4 that extra mile: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/25 https://github.com/geerlingguy/raspberry-pi-pcie-devices/iss... I'm most excited about a few with mini PCIe or M.2 slots on the bottom.
- exabrial 6y agowhat??? I _did not know_ PCI-E was exposed! I literally just had another discussion on HN a few weeks back when someone posted a 3d printed PI Cluster, and everyone lamented how lame storage latency was on a PI. I guess we're all pretty ignorant of the current offerings!
- parsimo2010 6y agoThis is pretty recent. The USB controller on the regular Pi 4 was sitting on the PCIE lines, so until the CM4 came out the only way to get to PCIE was to lose USB support and solder wires onto the traces on the PCB. A couple people did it but not many. Most people could get by with a USB 3.0 accessory and didn’t need to cut up their Pi. But now that the CM4 has a PCIE slot on the daughterboard it makes sense to see what is compatible.
- exabrial 6y agoAh, that was going to be my followup question. I'm having trouble finding a block diagram of the CM4 (and RPI4), if you happen to have a link, that'd be incredible. * Also, how the Ethernet PHY connect to the CPU and what are the implications?
- nuclearnice1 6y agoGo back and double check! kingosticks replied to you with exactly this link.
- varispeed 6y agoIs there a PCIe to ISA adapter? I'd love to hack some old cards from the 80s or is there any other way to connect ISA cards to RPi?