5 ms·

Good. IMHO, PCIe bandwidth is problematic on consumer devices. Can’t have multiple high bandwidth devices.

Night_Thastus 3y ago
Where is it problematic? On PCIe 4, you can run a high end graphics card and multiple SSDs over PCIe no problem. Bandwidth is not an issue there. If you're doing some server stuff, then maybe. But for normal consumers it's not an issue.
- michaelt 3y ago
  Modern consumer motherboards are in a bit of a weird place at the moment. Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. You want a motherboard with two 16x slots, to run a dual-GPU setup at full speed? Buy a threadripper. From a market segmentation perspective this makes sense from the processor manufacturers' point of view - get the people who want lots of IO onto your high-margin server/workstation product range. Back in the 1990s you needed to plug in a graphics card, a sound card, an ethernet card, maybe a separate wifi card, and so on, each of which would use exactly one slot - but these days other than the GPU, it's all integrated on the motherboard already, and the GPU might block 2 or 3 slots anyway. So modern consumer motherboards still sell OK despite having fewer slots than ever before!
  - Alupis 3y ago
    > Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. Maybe in the Intel World, this is true. Intel has always played games at consumer's expense... Here's the cheapest AMD AM5 socket ATX mobo sold by NewEgg, complete with two (2) 16x PCIe gen 4 slots[1]. Here's an AM5 mATX board with two (2) 16x PCIe gen 4 slots[2]. Come join team Red - the water is warm. [1] https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 [2] https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559 https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559
    - nolist_policy 3y ago
      Eh, on both boards the 2nd PCIe x16 slot only has 4 PCIe lanes - physically its an x16 slot and you can put x16 devices in it, but they will only be able to talk x4 speeds.
      - elabajaba 3y ago
        Even on high end AM5 motherboards, they'll only run at x8/x8 (or maybe x16/x4) if you populate both slots.
        loeg 3y ago
        PCIe 5.0 x8 has the same throughput as PCIe 4.0 x16, FWIW. So the x8/x8 configuration is very reasonable.
        elabajaba 3y ago
        Sure, but outside of a few SSDs (that aren't worth buying currently, need to wait for the 2nd gen pcie 5 controllers to tame their power draw), what consumer (or enthusiast) hardware is actually using PCIe 5 right now? If you slot 2 4090s into those x8/x8 5.0 slots, they'll "only" run at x8/x8 4.0 speeds.
      - 0x457 3y ago
        My board two slots PCI-E 5.0 that work as x8 to CPU if both used to x16 if only one used. As well as two PCI-E 5.0 x4 m.2 slots. To my understanding on a consumer market there isn't much that can utilize that much bandwidth. Current consumer GPUs tap out at 4.0 speeds, but use 5.0 (probably due to "more number more better" marketing). To benefit from 5.0 for NVMe it has to be very specific access patterns. It's only sad on Intel side because they want strong market segmentation and got away with it because AMD wasn't a competitor for years.
        DopplerSmell 3y ago
        I think there are a surprising number of people who would use two x16 slots (not the latest generation bandwidth requirements, just the electrical connection to support older products at full speed), one for a GPU, one for something like a networking or storage card. But it makes workstation and server class offerings weaker in comparison.
  - smileybarry 3y ago
    Some of the fancier PCIe 5.0 motherboards out there today actually do have two PCIe 16x slots -- one 5.0 and one 4.0. So you can run dual-GPU, though I don't see a reason to do this anymore except some niche GPU virtualization stuff. (multi-GPU proprietary tech is dead, DX12 mGPU never really took off, etc.)
  - ants_a 3y ago
    That's not the motherboards fault, the PCIe controller is on the CPU and needs to have corresponding I/O pinout on the socket. And even from the CPU manufacturer side, it's not artificial market segmentation. The I/O die alone on an EPYC/ThreadRipper Pro is more than double the die area of the CPU, GPU and I/O chips in a Ryzen 7 7800, and it's on a single chip which makes yields way worse. The socket and package need 1700 connections in one case and 6000 on the other. The high I/O CPU is way more expensive to manufacture, and on the lower end the prices start at ~1k for 512 GB/s of full duplex PCIe bandwidth and 12 memory channels.
  - numpad0 3y ago
    The 1x16/2x8 GPU port and 1x4 M.2 port goes directly to CPU, Rest of ports come through the Chipset/PCH/Northbridge/Southbridge/however you call it. The CPU-Chipset link is not real I/O bus coming from CPU die for quite some time. So motherboard brands can't just offer x16 to everyone. Breaking out full sized full speed buses was possible with PCI and PCI-X because those were real buses bridged to real CPU buses. That's not the case with PCIe. 0: https://images.anandtech.com/doci/17308/hlg-intel-prod-brief-w680chipset-block-diagram.jpg https://images.anandtech.com/doci/17308/hlg-intel-prod-brief... 1: https://www.lowyat.net/wp-content/uploads/2021/08/AMD-AM5-Diagram-chipset-1536x948.jpg https://www.lowyat.net/wp-content/uploads/2021/08/AMD-AM5-Di...
baobun 3y ago
You want more buses, not more bandwidth per bus.
technofiend 3y ago
As much as anything that's also due to Intel limiting lanes to on consumer CPUs keep from competing with their Xeon line. But yeah make each lane fast enough and it's less of a concern.
- throwaway48476 3y ago
  It would be if Broadcom didn't break the PLX PCIe switch market.
  - magicalhippo 3y ago
    Yeah, I mean for AM5 platform for example, it would have been awesome if some motherboards took x4 PCIe 5.0 lanes and turned them into x16 PCIe 3.0 lanes or something like that. That would benefit me way more than the current AM5 motherboards out there. Of course, as you note, the reason we don't see this is likely because the PCIe switch chips cost more than the rest of the motherboard.
  - foobiekr 3y ago
    details?
accrual 3y ago
It was kinda like that with regular PCI before PCIe as well. A single gigabit NIC could saturate the 133MB/s bus, so two or more wouldn't let you build a router with two full speed gigabit NICs for example.

Where is it problematic? On PCIe 4, you can run a high end graphics card and multiple SSDs over PCIe no problem. Bandwidth is not an issue there. If you're doing some server stuff, then maybe. But for normal consumers it's not an issue.

michaelt 3y ago
Modern consumer motherboards are in a bit of a weird place at the moment. Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. You want a motherboard with two 16x slots, to run a dual-GPU setup at full speed? Buy a threadripper. From a market segmentation perspective this makes sense from the processor manufacturers' point of view - get the people who want lots of IO onto your high-margin server/workstation product range. Back in the 1990s you needed to plug in a graphics card, a sound card, an ethernet card, maybe a separate wifi card, and so on, each of which would use exactly one slot - but these days other than the GPU, it's all integrated on the motherboard already, and the GPU might block 2 or 3 slots anyway. So modern consumer motherboards still sell OK despite having fewer slots than ever before!
- Alupis 3y ago
  > Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. Maybe in the Intel World, this is true. Intel has always played games at consumer's expense... Here's the cheapest AMD AM5 socket ATX mobo sold by NewEgg, complete with two (2) 16x PCIe gen 4 slots[1]. Here's an AM5 mATX board with two (2) 16x PCIe gen 4 slots[2]. Come join team Red - the water is warm. [1] https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 [2] https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559 https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559
  - nolist_policy 3y ago
    Eh, on both boards the 2nd PCIe x16 slot only has 4 PCIe lanes - physically its an x16 slot and you can put x16 devices in it, but they will only be able to talk x4 speeds.
    - elabajaba 3y ago
      Even on high end AM5 motherboards, they'll only run at x8/x8 (or maybe x16/x4) if you populate both slots.
      - loeg 3y ago
        PCIe 5.0 x8 has the same throughput as PCIe 4.0 x16, FWIW. So the x8/x8 configuration is very reasonable.
        elabajaba 3y ago
        Sure, but outside of a few SSDs (that aren't worth buying currently, need to wait for the 2nd gen pcie 5 controllers to tame their power draw), what consumer (or enthusiast) hardware is actually using PCIe 5 right now? If you slot 2 4090s into those x8/x8 5.0 slots, they'll "only" run at x8/x8 4.0 speeds.
    - 0x457 3y ago
      My board two slots PCI-E 5.0 that work as x8 to CPU if both used to x16 if only one used. As well as two PCI-E 5.0 x4 m.2 slots. To my understanding on a consumer market there isn't much that can utilize that much bandwidth. Current consumer GPUs tap out at 4.0 speeds, but use 5.0 (probably due to "more number more better" marketing). To benefit from 5.0 for NVMe it has to be very specific access patterns. It's only sad on Intel side because they want strong market segmentation and got away with it because AMD wasn't a competitor for years.
      - DopplerSmell 3y ago
        I think there are a surprising number of people who would use two x16 slots (not the latest generation bandwidth requirements, just the electrical connection to support older products at full speed), one for a GPU, one for something like a networking or storage card. But it makes workstation and server class offerings weaker in comparison.
- smileybarry 3y ago
  Some of the fancier PCIe 5.0 motherboards out there today actually do have two PCIe 16x slots -- one 5.0 and one 4.0. So you can run dual-GPU, though I don't see a reason to do this anymore except some niche GPU virtualization stuff. (multi-GPU proprietary tech is dead, DX12 mGPU never really took off, etc.)
- ants_a 3y ago
  That's not the motherboards fault, the PCIe controller is on the CPU and needs to have corresponding I/O pinout on the socket. And even from the CPU manufacturer side, it's not artificial market segmentation. The I/O die alone on an EPYC/ThreadRipper Pro is more than double the die area of the CPU, GPU and I/O chips in a Ryzen 7 7800, and it's on a single chip which makes yields way worse. The socket and package need 1700 connections in one case and 6000 on the other. The high I/O CPU is way more expensive to manufacture, and on the lower end the prices start at ~1k for 512 GB/s of full duplex PCIe bandwidth and 12 memory channels.
- numpad0 3y ago
  The 1x16/2x8 GPU port and 1x4 M.2 port goes directly to CPU, Rest of ports come through the Chipset/PCH/Northbridge/Southbridge/however you call it. The CPU-Chipset link is not real I/O bus coming from CPU die for quite some time. So motherboard brands can't just offer x16 to everyone. Breaking out full sized full speed buses was possible with PCI and PCI-X because those were real buses bridged to real CPU buses. That's not the case with PCIe. 0: https://images.anandtech.com/doci/17308/hlg-intel-prod-brief-w680chipset-block-diagram.jpg https://images.anandtech.com/doci/17308/hlg-intel-prod-brief... 1: https://www.lowyat.net/wp-content/uploads/2021/08/AMD-AM5-Diagram-chipset-1536x948.jpg https://www.lowyat.net/wp-content/uploads/2021/08/AMD-AM5-Di...

baobun 3y ago

You want more buses, not more bandwidth per bus.

technofiend 3y ago

As much as anything that's also due to Intel limiting lanes to on consumer CPUs keep from competing with their Xeon line. But yeah make each lane fast enough and it's less of a concern.

throwaway48476 3y ago
It would be if Broadcom didn't break the PLX PCIe switch market.
- magicalhippo 3y ago
  Yeah, I mean for AM5 platform for example, it would have been awesome if some motherboards took x4 PCIe 5.0 lanes and turned them into x16 PCIe 3.0 lanes or something like that. That would benefit me way more than the current AM5 motherboards out there. Of course, as you note, the reason we don't see this is likely because the PCIe switch chips cost more than the rest of the motherboard.
- foobiekr 3y ago
  details?

accrual 3y ago

It was kinda like that with regular PCI before PCIe as well. A single gigabit NIC could saturate the 133MB/s bus, so two or more wouldn't let you build a router with two full speed gigabit NICs for example.