3 ms·

Makes me wonder what the highest perf would look like out of an arbitrary hypothetical multi-socket Apple Silicon system, vs an arbitrary multi-socket x86 syste

by derefr 4y ago

Makes me wonder what the highest perf would look like out of an arbitrary hypothetical multi-socket Apple Silicon system, vs an arbitrary multi-socket x86 system; where the only constraints for both systems are that the boards have a fixed power budget. (I.e. "who wins in a map/reduce task: a board that spends 1000W to power 4 Xeons, or a board that spends 1000W to power 20 M2 Ultras?") Too bad there are no bare/unsoldered Apple Silicon chips to try building such boards around. I'm sure, if there were, you'd find them all over AliExpress. I'd also be curious which of those two boards would have a higher BOM!

bee_rider 4y ago
What’s the fastest network for a M2 Ultra? Can we run some MPI codes?
- piperswe 4y ago
  You could probably run a QDR+QSFP Infiniband card at around 32Gbps (minus overhead) through an external "GPU" enclosure. I don't see why MPI wouldn't work on Asahi Linux with such a setup once there's Thunderbolt support.
  - jabl 4y ago
    QDR Infiniband is, like, 2007 level tech. Today we have NDR Infiniband where a typical 4xHCA gets you 400 GB/s. Seems like a hypothetical Mac cluster would be severely limited by this compared to the typical x86 based server clusters.
    - bee_rider 4y ago
      I’m sure such a switch would be serving some pretty beefy nodes, though, right? Maybe the compute:communication can be held constant with less-powerful Mac mini nodes?
    - piperswe 4y ago
      The problem is that the Thunderbolt ports don't support that bandwidth, and there isn't any other way to connect PCIe peripherals.
  - bee_rider 4y ago
    It is apparently possible to do networking over some Thunderbolt interfaces, would it be possible to connect the devices over Thunderbolt to one another directly? Four ports each, so form a mesh! I guess TB4 can go up to 40Gbps, although it sounds like there’s a bit of overhead when using it as a network, and also I have no idea if there’s some hub-like bottleneck inside the chip…
dezgeg 4y ago
The RAM on the M1 is soldered to the SoC package, so building a multi-socket system with shared RAM would most likely be impossible.

by derefr 4y ago

bee_rider 4y ago

What’s the fastest network for a M2 Ultra? Can we run some MPI codes?

piperswe 4y ago
You could probably run a QDR+QSFP Infiniband card at around 32Gbps (minus overhead) through an external "GPU" enclosure. I don't see why MPI wouldn't work on Asahi Linux with such a setup once there's Thunderbolt support.
- jabl 4y ago
  QDR Infiniband is, like, 2007 level tech. Today we have NDR Infiniband where a typical 4xHCA gets you 400 GB/s. Seems like a hypothetical Mac cluster would be severely limited by this compared to the typical x86 based server clusters.
  - bee_rider 4y ago
    I’m sure such a switch would be serving some pretty beefy nodes, though, right? Maybe the compute:communication can be held constant with less-powerful Mac mini nodes?
  - piperswe 4y ago
    The problem is that the Thunderbolt ports don't support that bandwidth, and there isn't any other way to connect PCIe peripherals.
- bee_rider 4y ago
  It is apparently possible to do networking over some Thunderbolt interfaces, would it be possible to connect the devices over Thunderbolt to one another directly? Four ports each, so form a mesh! I guess TB4 can go up to 40Gbps, although it sounds like there’s a bit of overhead when using it as a network, and also I have no idea if there’s some hub-like bottleneck inside the chip…

dezgeg 4y ago

The RAM on the M1 is soldered to the SoC package, so building a multi-socket system with shared RAM would most likely be impossible.