5 ms·
One neat thing about PCIe is that every PCIe compliant device must support using less lanes than it can. So your GPU which would like to work with x16 must supp
by magicalhippo 5y ago
One neat thing about PCIe is that every PCIe compliant device must support using less lanes than it can. So your GPU which would like to work with x16 must support x1 etc.
So in theory one could install a GPU or HBA in one of those x1 slots on the motherboard if one isn't terribly concerned about bandwidth. Except most of the motherboards use x1 connectors which doesn't have an "open back" for larger cards... so much for that[1].
Another thing I didn't see it mentioning was this. Say you have a x16 slot on your motherboard, but you really want to install 4 NVME m.2 drives. Well each of those need a x4, but you have an x16 so per above you should be fine right, just plop in a "dumb" four socket m.2 PCIe card?
Well, no, not unless the motherboard supports bifurcation[2]. If not, you'll need a card with a PCIe switch which can turn those four x4's into one x16.
[1]: yes I know about risers
[2]: https://www.10gtek.com/new-1414 https://www.10gtek.com/new-1414
- FractalParadigm 5y agoFor point [1] it's also trivial to file or Dremel the slot to be open-backed, if you're careful. I've done this successfully a handful of times, unsuccessfully a few more than that...
- magicalhippo 5y agoTrue, I'm just really annoyed an open-back socket isn't the default choice for the manufacturers. Not like I enjoy using a Dremel on my motherboard...
- dogma1138 5y agoYou’ll find many cases where lack of open back slots usually also means there will be components behind the slot that will interfere with a full/half length card. Also don’t use a Dremel or even a file a hobby knife or flush cutters work just fine. If you have a steady hand then a hot knife attachment for a soldering iron is fine too. The cutting wheel on a dremel tool can run off and the dust it produces can be conductive and short something.
- krisoft 5y agoLet me get you straight: you say that the process is trivial, yet you admit to have a worse than 50% failure ratio? What does “unsuccessfully” mean in this case? Is the part completely ruined? At what failure rate would you consider a process not-trivial?
- AboveTheGame 5y agoThere is a useful lesson here: most people don’t use words to mean anything other than what they believe they are describing. Trivial to him means it doesn’t take a lot of time, which is how he uses the word, in the context of time. Now, he’s unaware that he’s wrong and he is also unaware that his misuse of the word means his explanation is irrelevant to anyone who doesn’t already understand him! In effect, he says nothing and you respond with confusion asking why you needed to understand what he didn’t! (People are very very good with pretending at intelligence and bad at the practice of it)
- kymaz 5y agoIt's trivial to take a rotary tool to a piece of plastic, but now you have plastic dust and shavings all over the place. like on the PCIE contacts. You could also use a razor blade or other such manual cutting devices...
- FractalParadigm 5y agoAdmittedly it was more of a tongue-in-cheek joke with some fact - I've done it successfully 4 or 5 times on systems meant for "production" only after testing on EoL/scheduled for the bin hardware with mixed results - at least two I hit data pins and rendered the port unusable, one the Dremel slipped and ended up cutting a slot through the board itself, and another one you would have sworn was perfect but just didn't go (probably something shorted out I couldn't physically see). I did three good ones after those 5 or 6 failures, at which point a friend wanted me to do all the x1 ports on his board to fit x4 NICs he had, not caring if it was successful or not (he had backup hardware just in case, which thanfully wasn't needed), followed by another friend with a server he wanted to put a GPU in. If someone handed me a board today I'd feel fairly comfortable I could do it without issue; I still wouldn't exactly recommend it to the faint-of-heart or those without steady hands, but once you kinda 'figure out' the right way to do it with the right tools, it's pretty straightforward.
- mnd999 5y agoThe fail0verflow team that got Linux to run on the PS4 ran a single, very slow PCIe lane over RS232 as I recall so that they could intercept the traffic. It’s very tolerant.
- myself248 5y agoYesssss, one of my favorite hardware-hacking talks ever: https://media.ccc.de/v/33c3-7946-console_hacking_2016#t=375 https://media.ccc.de/v/33c3-7946-console_hacking_2016#t=375
- tonyarkles 5y agohttps://fail0verflow.com/blog/2016/console-hacking-2016-postscript/ https://fail0verflow.com/blog/2016/console-hacking-2016-post... I think this is the talk that goes through their PCIe intercept stuff. Absolutely hilarious to me; I believe they were running one direction at 115.2kbps and everything was just a-ok about it.
- Causality1 5y agoPCIe compliant device must support using less lanes than it can Is that true in practice or is it like USB where manufacturers can wildly violate specs and just expect consumers to deal with it?
- f001 5y agoIt is true in practice. The older style of mining rigs for certain types of cryptocurrency rely on a ton of gpus (6-18) connected via risers to every available pci-e slot on a motherboard, most of them being 1x slots. There are special motherboards for this that don’t even come with 16x slots, instead breaking that out into 16 1x slots to allow for more gpus to be connected.
- giobox 5y agoIt generally always works in practice. Common real-world example is “External GPUs” connecting via thunderbolt: these are actually just generic PCI express 4x slots on a cable (4x is max throughout thunderbolt 3 achieves, can’t remember what cap is on new thunderbolt 4…). Although graphics cards always generally should go in a 16x slot for best performance, they all work in the external 4x, even the latest most high end power/bandwidth hungry GPU parts. performance may suffer in some workloads though due to reduced bandwidth of the 4x slot though. You can stick any PCI express device in an external GPU enclosure, will pretty much “just work” despite being a 4x slot - these things are really external PCIe enclosures despite the GPU focus.
- Dylan16807 5y agoIn terms of PCIe 3 lanes, the max on thunderbolt 3 is 2.75x, and the max on thunderbolt 4 is 4x. Out of a roughly-5x total bandwidth. Why was 3 capped? Nobody seems to know.
- novok 5y agoAFAIK thunderbolt has an overhead that it adds to plain old PCIe, making it slower in practice. I kind of wish it was just plain PCIe, you could directly connect cheap-ish pcie adapters to it then.
- 5y ago
- cesarb 5y ago> So in theory one could install a GPU or HBA in one of those x1 slots on the motherboard if one isn't terribly concerned about bandwidth. Except most of the motherboards use x1 connectors which doesn't have an "open back" for larger cards... so much for that[1]. It's not so simple, there's another factor other than the number of lanes: power. According to Wikipedia (https://en.wikipedia.org/wiki/PCI_Express#Power https://en.wikipedia.org/wiki/PCI_Express#Power), a x1 slot is limited to 25W on the 12V pins, while a x16 slow has a higher limit of 66W; the power pins are the same, but the maximum current is higher (2.1A vs 5.5A).
- magicalhippo 5y agoFair point, I didn't think of that since my primary wants for a x1 port has been low-power stuff.
- secondcoming 5y agoI have a wifi6 card that only works in the x1 slot, for some really annoying reason
- amenghra 5y agoI have come across devices which would only work on specific slots. There’s clearly a gap between theory and practice…
- pixl97 5y agoLots of BIOS/efi just really suck. On the enterprise side of things loading up a board with bootable raid and 10g cards will lead to situations where the raid card needs to go topologically first or early so the raid bios has memory to load.
- bofh23 5y agoThere’s a nice photo of an open back PCIe x1 connector here: File:PCIe J1900 SoC ITX Mainboard IMG 1820.JPG - Wikipedia https://en.wikipedia.org/wiki/File:PCIe_J1900_SoC_ITX_Mainboard_IMG_1820.JPG https://en.wikipedia.org/wiki/File:PCIe_J1900_SoC_ITX_Mainbo...
- deaddodo 5y agoOpen back and closed back PCIe connectors are electrically exactly the same. There are plenty of instructions on the net for grinding out the backplane to allow physically larger cards[1]. 1 - https://youtu.be/fZed5r9tXHQ https://youtu.be/fZed5r9tXHQ
- rjsw 5y agoSome dev boards have a PCIe slot with an "open back".
- StillBored 5y ago"x1 connectors which doesn't have an "open back" for larger cards..." Because they bothered to read the PCIe spec. Open back slots are against the PCIe specification ("up plugging" is the term IIRC). That is for two (three?) reasons, first its a mechanical thing, if you noticed plugging a big heavy x16 GPU into a x1 slot is an exercise in getting it just right. Any bump/whatever will move the card into a bad angle, even if its screwed into the back panel. Second, in the past, with older pcie specs the amount of current a slot was required to supply was dependent on its size. So a big x16 card might need the full slot rating while a smaller slot might only provide a smaller amount of current. (IIRC the limits were something like 75W for a x16 and only 25W for a x1). Thirdly, down plugging _IS_ part of the spec and the correct way to handle this situation. That means the MB vendor provides a larger, say x16, mechanical slot which is provides a fraction of the signal lanes, say x4. So, the vendors which were providing cut back slots were either ignorant of the spec, too cheap to pay the couple extra cents for the larger connector, or had some fundamental constrain on providing it and were willing to provide a non-conformant part (generally unlikely). I've seen these cut back slots a few times in parts of the arm ecosystem where the vendor can't be bothered to read the spec much less implement it correctly. This is also how one has the pile of problems that are frequently seen in the SBC market where the boards won't actually work with some PCIe card, USB device, whatever because the HW/SW is only implementing the convenient parts the relevant standard.
- axiolite 5y ago> One neat thing about PCIe is that every PCIe compliant device must support using less lanes than it can. So your GPU which would like to work with x16 must support x1 etc. I tried this with a SAS controller. They make 16-port SAS PCI-e x8 cards, so on a x1 slot I should get full speed for 1-2 SAS devices, and the SAS drive(s) will only slow-down if I add more... right? Wrong. Even with a single device, you get about 1/8th the speed you should. So, yeah, PCI-e cards properly downgrade to fewer lanes, but not necessarily in a very useful way.