7 ms·

That and refusing to up the number of directly connected PCI lanes. The latest chips support 40 lanes but some of them are outboard of the CPU and connected vi

That and refusing to up the number of directly connected PCI lanes. The latest chips support 40 lanes but some of them are outboard of the CPU and connected via a controller. High speed cards like a 10 gigabit Mellanox can use 8 PCI lanes all by itself. Throw in some M.2 devices and you're out of directly connected lanes.

CyberDildonics 9y ago
10 gigabits is about 1.2 gigabytes. PCIe bandwidth is about 1 gigabyte per lane. A 10gb card does not need 8 PCIe lanes.
- kogepathic 9y ago
  > PCIe bandwidth is about 1 gigabyte per lane Per lane (unidirectional) bandwidth by PCIe revision [0]: PCIe 1.0: 250MB/s PCIe 2.0: 500MB/s PCIe 3.0: 985MB/s PCIe 4.0: 1.969GB/s PCIe 5.0: 3.9GB/s > A 10gb card does not need 8 PCIe lanes. It depends on the PCIe revision(s) the card supports. If the card needs to support 10GBit on a PCIe 1.0 link, then it will need an x8. The bandwidth of PCIe 1.0 is ~250MB/s per x1 link, but there is protocol overhead and 4 lanes doesn't provide enough bandwidth, so you'd have to use an x8 (since x6 keyed cards are not a thing). The most popular cards with enthusiasts are refurb Mellanox cards, which are PCIe 2.0 but support PCIe 1.1, so they need to utilize PCIe x8 to provide full speed 10GBit: PCI Express 2.0 x8 (1.1 compatible) [1] [0] https://en.wikipedia.org/wiki/PCI-Express https://en.wikipedia.org/wiki/PCI-Express [1] http://www.mellanox.com/page/products_dyn?product_family=20&mtag=connectx_en http://www.mellanox.com/page/products_dyn?product_family=20&...
  - CyberDildonics 9y ago
    PCIe 3.0 was finalized a decade ago. Equating old cards to a technical limitation is ridiculous.
    - technofiend 9y ago
      Funnily enough it was the mellanox I had in mind.
    - kogepathic 9y ago
      > PCIe 3.0 was finalized a decade ago. Equating old cards to a technical limitation is ridiculous. No, it's not. Not when the cards people are using are targeted at PCIe 2.0/1.1. Since the context was 10GBit cards, most people are using the Mellanox card I linked to, which is PCIe 2.0. Whine about the age of the standard all you want, what's on the market today and in use by a majority of home users is PCIe x8 for 10GBit.
      - CyberDildonics 9y ago
        > High speed cards like a 10 gigabit Mellanox can use 8 PCI lanes all by itself. Throw in some M.2 devices and you're out of directly connected lanes. The original post was saying there is a shortage of PCI lanes because of situations like a 10gb network card using 8 lanes. The truth of the matter is that if someone is running out of lanes in that situation, it is because of exceptionally inefficient use of the lanes, not because 40 lanes is as limiting as what is being implied.
        kogepathic 9y ago
        > it is because of exceptionally inefficient use of the lanes No, it's because the customer is installing the cards they want to use, and those cards are consuming PCIe lanes. A system with more PCIe lanes lets the customer install more of whatever they want. The PCIe revision doesn't matter, and neither does the "efficiency" of the cards they're installing. More lanes means the customer can install more of whatever they want/need to install. Sure, you could spend more on a 10GBit card that supports a newer PCIe revision and doesn't use 8 lanes, or just go and buy an AMD system which costs less than Intel and provides 64 PCIe lanes anyway.
        CyberDildonics 9y ago

CyberDildonics 9y ago

10 gigabits is about 1.2 gigabytes. PCIe bandwidth is about 1 gigabyte per lane. A 10gb card does not need 8 PCIe lanes.

kogepathic 9y ago
> PCIe bandwidth is about 1 gigabyte per lane Per lane (unidirectional) bandwidth by PCIe revision [0]: PCIe 1.0: 250MB/s PCIe 2.0: 500MB/s PCIe 3.0: 985MB/s PCIe 4.0: 1.969GB/s PCIe 5.0: 3.9GB/s > A 10gb card does not need 8 PCIe lanes. It depends on the PCIe revision(s) the card supports. If the card needs to support 10GBit on a PCIe 1.0 link, then it will need an x8. The bandwidth of PCIe 1.0 is ~250MB/s per x1 link, but there is protocol overhead and 4 lanes doesn't provide enough bandwidth, so you'd have to use an x8 (since x6 keyed cards are not a thing). The most popular cards with enthusiasts are refurb Mellanox cards, which are PCIe 2.0 but support PCIe 1.1, so they need to utilize PCIe x8 to provide full speed 10GBit: PCI Express 2.0 x8 (1.1 compatible) [1] [0] https://en.wikipedia.org/wiki/PCI-Express https://en.wikipedia.org/wiki/PCI-Express [1] http://www.mellanox.com/page/products_dyn?product_family=20&mtag=connectx_en http://www.mellanox.com/page/products_dyn?product_family=20&...
- CyberDildonics 9y ago
  PCIe 3.0 was finalized a decade ago. Equating old cards to a technical limitation is ridiculous.
  - technofiend 9y ago
    Funnily enough it was the mellanox I had in mind.
  - kogepathic 9y ago
    > PCIe 3.0 was finalized a decade ago. Equating old cards to a technical limitation is ridiculous. No, it's not. Not when the cards people are using are targeted at PCIe 2.0/1.1. Since the context was 10GBit cards, most people are using the Mellanox card I linked to, which is PCIe 2.0. Whine about the age of the standard all you want, what's on the market today and in use by a majority of home users is PCIe x8 for 10GBit.
    - CyberDildonics 9y ago
      > High speed cards like a 10 gigabit Mellanox can use 8 PCI lanes all by itself. Throw in some M.2 devices and you're out of directly connected lanes. The original post was saying there is a shortage of PCI lanes because of situations like a 10gb network card using 8 lanes. The truth of the matter is that if someone is running out of lanes in that situation, it is because of exceptionally inefficient use of the lanes, not because 40 lanes is as limiting as what is being implied.
      - kogepathic 9y ago
        > it is because of exceptionally inefficient use of the lanes No, it's because the customer is installing the cards they want to use, and those cards are consuming PCIe lanes. A system with more PCIe lanes lets the customer install more of whatever they want. The PCIe revision doesn't matter, and neither does the "efficiency" of the cards they're installing. More lanes means the customer can install more of whatever they want/need to install. Sure, you could spend more on a 10GBit card that supports a newer PCIe revision and doesn't use 8 lanes, or just go and buy an AMD system which costs less than Intel and provides 64 PCIe lanes anyway.
        CyberDildonics 9y ago