2 ms·

Alas PCIe switches were starting to get affordable before the great chipmaker consolidation of 2015-2019 ("and seemingly overnight the cost of these switches in

Alas PCIe switches were starting to get affordable before the great chipmaker consolidation of 2015-2019 ("and seemingly overnight the cost of these switches increased three-fold"[1]). Now most every PCIe switch is >$6/lane. Even just the 12 lanes here, on an old PCIe generation, is going to be a $72 chip, perhaps more (article: PEX8619 Broadcom PCIe switch, $129, 16 lanes, $8/lane, no stock. Compare versus Microchip nee PMC-Sierra Switchtec such as PM40028B1 at $6/lane[2]). It's a cool idea. But wow, for $435, sure feels one ought consider going for an x86 pc. The form factor is cool though. It feels insane to hope for but I'm really hoping that by the time 2025 rolls around we see some modest affordable arm chips that have Thunderbolt built in. Perhaps even multiple Thunderbolt. If nothing else, there better be some decent PCIe available by then- 8x PCIe 5.0 or better. ARM needs to start doing bandwidth, some day (in places other than server class chips); it's been on USB3.0 for a long long time. Thunderbolt + PCIe would be a great boon for general usability & connectivity. Now that Intel has made it royalty free, now that USB4 already incorporates most of the hard part (packet based switching), it might finally be the time for some real PCIe on chip. [1] https://www.anandtech.com/show/15821/microchips-new-pcie-40-pcie-switches-100-lanes-174-gbps https://www.anandtech.com/show/15821/microchips-new-pcie-40-... [2] https://www.mouser.com/c/?marcom=100722352 https://www.mouser.com/c/?marcom=100722352

MisterTea 5y ago
> If nothing else, there better be some decent PCIe available by then- 8x PCIe 5.0 or better. ARM needs to start doing bandwidth, some day (in places other than server class chips); it's been on USB3.0 for a long long time. Its not up to Arm to attach PCIe lanes but the chip makers who have been competing with x86 which already offer plenty of PCIe lanes. So far, only apple has bothered to make an Arm CPU which can compete with "PC" hardware. Hopefully AMD will offer something similar. > Thunderbolt + PCIe would be a great boon for general usability & connectivity. Now that Intel has made it royalty free, now that USB4 already incorporates most of the hard part (packet based switching), it might finally be the time for some real PCIe on chip. USB4 is based on Thunderbolt 3. Given that Thunderbolt 3 is a mess, I don't have high hopes. Look at RapidIO or 1394 for examples of better thought out interconnects and protocols. PCIe cant connect CPU's together in a NUMA system whereas RapiIO can. RapidIO also uses Ethernet physical layers so there's parallel development with Ethernet. I honestly don't like USB or contraptions invented by Intel that are invariably consumer grade with various extensions to satisfy other industries, bolted on.
- NovemberWhiskey 5y ago
  >So far, only apple has bothered to make an Arm CPU which can compete with "PC" hardware. Also Amazon: https://aws.amazon.com/ec2/graviton/ https://aws.amazon.com/ec2/graviton/
  - MisterTea 5y ago
    Those are server processors, I was thinking more along the lines of laptop/desktop performance. Other examples of Arm server CPU's are from Ampere and AMD's discontinued Opteron A series.
- rektide 5y ago
  i thought it was pretty clear i meant the consumer arm ecosystem in general but thanks for chiming in & clarifying. yup. usb4's ability to transport a variety of protocols across the switched network seems perfect. reviews like the Plugable tb4 hub show video and data intermixing nicely[1]. the complexity seems scary but in my mind it feels utterly & completely justified, and it builds atop the world we have & makes it better, quietly fixes the underpinnings, rather than insisting on re-invention. thunderbolt hasn't been flawless but it's been a niche product, a luxury product, and we'll quickly gain the experience to make it as pleasant bulletproof & flawless as one might hope for. already, reviews like the one i cited don't come with caveats or speed bumps: they show a working, satisfying consumer experience. there's a lot of afraidness of thunderbolt & usb4, but i for one think we ought to dial down our resistances & afraidness. that said, i also think the door is totally open to new interconnects emerging. i'd love to see cxl or gen-z (recently folded into cxl) or opencapi or rapidio or whatever become consumerized. [1] https://www.anandtech.com/show/16964/plugable-tbt4hub3c-thunderbolt-4-hub-capsule-review https://www.anandtech.com/show/16964/plugable-tbt4hub3c-thun...
scottlamb 5y ago
> It feels insane to hope for but I'm really hoping that by the time 2025 rolls around we see some modest affordable arm chips that have Thunderbolt built in. Perhaps even multiple Thunderbolt. If nothing else, there better be some decent PCIe available by then- 8x PCIe 5.0 or better. In the shorter term, I'm hoping the RK3588 actually comes out, is as good as promised, and has a reasonable price. [1] 4 lanes of PCIe 3.0 + 3 lanes of PCIe 2.0 + 2x USB 3.1 + dual gigabit Ethernet total at well over 10X the bandwidth of the Pi's single PCIe 2.0 lane. Also things I care about for my NVR project: faster video decoding, built-in TPU, and SATA (which I didn't count above because it's multiplexed with the PCIe 2.0). I don't think the 4x Cortex-A76 + 4x Cortex-A55 is in the same league as the Apple M1, but still quite a bit faster than the Pi4's 4x Cortex-A72. [1] https://www.cnx-software.com/2020/11/26/rockchip-rk3588-specifications-revealed-8k-video-6-tops-npu-pcie-3-0-up-to-32gb-ram/ https://www.cnx-software.com/2020/11/26/rockchip-rk3588-spec...
- CameronNemo 5y ago
  I don't think the 4x Cortex-A76 + 4x Cortex-A55 is in the same league as the Apple M1, but still quite a bit faster than the Pi4's 4x Cortex-A72. IMO the node used is more important than the core design. Rockchip claims they will be using an 8nm node for the rk3588. And I think the rk3399 and Pi4 are using 28nm nodes right now. So if this chip materializes it could be much faster (compared to Linux oriented ARM boards of today).

MisterTea 5y ago

> If nothing else, there better be some decent PCIe available by then- 8x PCIe 5.0 or better. ARM needs to start doing bandwidth, some day (in places other than server class chips); it's been on USB3.0 for a long long time. Its not up to Arm to attach PCIe lanes but the chip makers who have been competing with x86 which already offer plenty of PCIe lanes. So far, only apple has bothered to make an Arm CPU which can compete with "PC" hardware. Hopefully AMD will offer something similar. > Thunderbolt + PCIe would be a great boon for general usability & connectivity. Now that Intel has made it royalty free, now that USB4 already incorporates most of the hard part (packet based switching), it might finally be the time for some real PCIe on chip. USB4 is based on Thunderbolt 3. Given that Thunderbolt 3 is a mess, I don't have high hopes. Look at RapidIO or 1394 for examples of better thought out interconnects and protocols. PCIe cant connect CPU's together in a NUMA system whereas RapiIO can. RapidIO also uses Ethernet physical layers so there's parallel development with Ethernet. I honestly don't like USB or contraptions invented by Intel that are invariably consumer grade with various extensions to satisfy other industries, bolted on.

NovemberWhiskey 5y ago
>So far, only apple has bothered to make an Arm CPU which can compete with "PC" hardware. Also Amazon: https://aws.amazon.com/ec2/graviton/ https://aws.amazon.com/ec2/graviton/
- MisterTea 5y ago
  Those are server processors, I was thinking more along the lines of laptop/desktop performance. Other examples of Arm server CPU's are from Ampere and AMD's discontinued Opteron A series.
rektide 5y ago
i thought it was pretty clear i meant the consumer arm ecosystem in general but thanks for chiming in & clarifying. yup. usb4's ability to transport a variety of protocols across the switched network seems perfect. reviews like the Plugable tb4 hub show video and data intermixing nicely[1]. the complexity seems scary but in my mind it feels utterly & completely justified, and it builds atop the world we have & makes it better, quietly fixes the underpinnings, rather than insisting on re-invention. thunderbolt hasn't been flawless but it's been a niche product, a luxury product, and we'll quickly gain the experience to make it as pleasant bulletproof & flawless as one might hope for. already, reviews like the one i cited don't come with caveats or speed bumps: they show a working, satisfying consumer experience. there's a lot of afraidness of thunderbolt & usb4, but i for one think we ought to dial down our resistances & afraidness. that said, i also think the door is totally open to new interconnects emerging. i'd love to see cxl or gen-z (recently folded into cxl) or opencapi or rapidio or whatever become consumerized. [1] https://www.anandtech.com/show/16964/plugable-tbt4hub3c-thunderbolt-4-hub-capsule-review https://www.anandtech.com/show/16964/plugable-tbt4hub3c-thun...

scottlamb 5y ago

> It feels insane to hope for but I'm really hoping that by the time 2025 rolls around we see some modest affordable arm chips that have Thunderbolt built in. Perhaps even multiple Thunderbolt. If nothing else, there better be some decent PCIe available by then- 8x PCIe 5.0 or better. In the shorter term, I'm hoping the RK3588 actually comes out, is as good as promised, and has a reasonable price. [1] 4 lanes of PCIe 3.0 + 3 lanes of PCIe 2.0 + 2x USB 3.1 + dual gigabit Ethernet total at well over 10X the bandwidth of the Pi's single PCIe 2.0 lane. Also things I care about for my NVR project: faster video decoding, built-in TPU, and SATA (which I didn't count above because it's multiplexed with the PCIe 2.0). I don't think the 4x Cortex-A76 + 4x Cortex-A55 is in the same league as the Apple M1, but still quite a bit faster than the Pi4's 4x Cortex-A72. [1] https://www.cnx-software.com/2020/11/26/rockchip-rk3588-specifications-revealed-8k-video-6-tops-npu-pcie-3-0-up-to-32gb-ram/ https://www.cnx-software.com/2020/11/26/rockchip-rk3588-spec...

CameronNemo 5y ago
I don't think the 4x Cortex-A76 + 4x Cortex-A55 is in the same league as the Apple M1, but still quite a bit faster than the Pi4's 4x Cortex-A72. IMO the node used is more important than the core design. Rockchip claims they will be using an 8nm node for the rk3588. And I think the rk3399 and Pi4 are using 28nm nodes right now. So if this chip materializes it could be much faster (compared to Linux oriented ARM boards of today).