3 ms·

Don't worry, the chip you are searching is branded ExpressFabric and sold/made by Broadcom. It's AFAIK only PCIe4, and you get ones with up to iirc 96 lanes per

by namibj 8y ago

Don't worry, the chip you are searching is branded ExpressFabric and sold/made by Broadcom. It's AFAIK only PCIe4, and you get ones with up to iirc 96 lanes per chip. They support to be connected into a mesh, even adding shared PCIe devices like network cards, GPUs, NVMe and SAS/SATA HBAs is possible, though you might have difficulties with the GPUs in particular. The others, if you are careful when buying, should do well if they can handle SR-IOV, as (from their view), the mesh switch they are attached to is the root, and the computers attached to the mesh take the place a VM would normally occupy. If you could get the smallest chip they make, it should not be hard to get a PCB made for it that fans out to e.g. a set of USB-C receptacles. One such receptacle can handle 2 PCIe3 lanes, and some power/USB-2 on the side. The main reason being the relatively low price, compared to e.g. mini SAS HD, which has a similar density and twice the lanes per connector, but also a slightly higher target impedance at the upper end of the PCIe spec, whereas USB-C is specifically targeting the perfect PCIe impedance.

justinclift 8y ago
A quick search on Digikey shows several entries for ExpressFabric which seem like what you mean: https://www.digikey.com/products/en?keywords=expressfabric https://www.digikey.com/products/en?keywords=expressfabric They all have a part status of "Discontinued at Digi-Key" though. Not seeing any further info of where they _are_ available either. :/
- namibj 8y ago
  You found the chips I was referring to. If you want them, you should probably ask Broadcom, as they should either be able to point you to where you can still buy them, or to a replacement. And in case they don't help you, you can still ask sales at competitors, as some might have something suitable. This certainly looks like they'd do a run for you if you want some, albeit with a more significant MOQ (probably in increments of one wafer, with the best price to you if you agree to take all good dies of those they make for you). According to their website [0], they are still active products. It just seems that no one really wants to buy them, due them being somewhat weird. There is a reference platform [1], which is a 32-port, as far as I can tell PCIe3 x4 on QSFP ports, 1U TOR switch. It goes with 2-port cards that go into the servers and contain re-timers. Broadcom claims one can connect the switch with the server cards using optics or copper, but the switch is $11k at mouser. The technology apparently offers virtual Ethernet NICs and 8 QOS classes. Did you want to buy such technology? [0]: https://www.broadcom.com/products/pcie-switches-bridges/expressfabric/#tab-PCIe1 https://www.broadcom.com/products/pcie-switches-bridges/expr... [1]: https://www.broadcom.com/applications/datacenter-networking/expressfabric https://www.broadcom.com/applications/datacenter-networking/... (bottom of the page, grep "ExpressFabric Reference Platform")
  - justinclift 8y ago
    > Did you want to buy such technology? Nope, just merely curious For most of the use cases I can think of that these would suit, Infiniband seems like it would also work and that's fairly widely available already. That being said, there's probably use cases this would suit better. They're just not coming to mind easily. :)
    - namibj 8y ago
      Cheaper than Infiniband, I assume. Even if you buy the TOR switch, from what I can tell. Also, the latency is much smaller than for Infiniband, and it is nice to e.g. combine blades with NICs or similar configurations, without the NICs being special multi-host ones like Mellanox offers.

by namibj 8y ago

justinclift 8y ago

A quick search on Digikey shows several entries for ExpressFabric which seem like what you mean: https://www.digikey.com/products/en?keywords=expressfabric https://www.digikey.com/products/en?keywords=expressfabric They all have a part status of "Discontinued at Digi-Key" though. Not seeing any further info of where they _are_ available either. :/

namibj 8y ago
You found the chips I was referring to. If you want them, you should probably ask Broadcom, as they should either be able to point you to where you can still buy them, or to a replacement. And in case they don't help you, you can still ask sales at competitors, as some might have something suitable. This certainly looks like they'd do a run for you if you want some, albeit with a more significant MOQ (probably in increments of one wafer, with the best price to you if you agree to take all good dies of those they make for you). According to their website [0], they are still active products. It just seems that no one really wants to buy them, due them being somewhat weird. There is a reference platform [1], which is a 32-port, as far as I can tell PCIe3 x4 on QSFP ports, 1U TOR switch. It goes with 2-port cards that go into the servers and contain re-timers. Broadcom claims one can connect the switch with the server cards using optics or copper, but the switch is $11k at mouser. The technology apparently offers virtual Ethernet NICs and 8 QOS classes. Did you want to buy such technology? [0]: https://www.broadcom.com/products/pcie-switches-bridges/expressfabric/#tab-PCIe1 https://www.broadcom.com/products/pcie-switches-bridges/expr... [1]: https://www.broadcom.com/applications/datacenter-networking/expressfabric https://www.broadcom.com/applications/datacenter-networking/... (bottom of the page, grep "ExpressFabric Reference Platform")
- justinclift 8y ago
  > Did you want to buy such technology? Nope, just merely curious For most of the use cases I can think of that these would suit, Infiniband seems like it would also work and that's fairly widely available already. That being said, there's probably use cases this would suit better. They're just not coming to mind easily. :)
  - namibj 8y ago
    Cheaper than Infiniband, I assume. Even if you buy the TOR switch, from what I can tell. Also, the latency is much smaller than for Infiniband, and it is nice to e.g. combine blades with NICs or similar configurations, without the NICs being special multi-host ones like Mellanox offers.