3 ms·
This is a good point. They have 8 channels of DDR4-3200, which is 204 GB/s, to be shared across 128 cores. That's half the total memory bandwidth of the M1 Max,
by marcan_42 5y ago
This is a good point. They have 8 channels of DDR4-3200, which is 204 GB/s, to be shared across 128 cores. That's half the total memory bandwidth of the M1 Max, at 400 GB/s, and it only has 10 cores and a (decently beefy) GPU.
A single M1 P-core can saturate 70GB/s of memory bandwidth, and not even at max clock. If these 128 cores were individually as good as the M1 cores, they'd need 350 channels of DDR4-3200 RAM to get that nonblocking per-core memory bandwidth. The M1 Max can't achieve that, ending up at somewhere around 28GB/s per core with all P-cores competing, but even then this 128-core CPU would need 140 channels of DDR4-3200 to achieve the same memory bandwidth per core. So I imagine this kind of CPU will scale well to compute-heavy workloads that mostly stay in cache, but rather poorly for throughput-heavy workloads that stream data to/from RAM very quickly. The RAM is more than 10x oversubscribed; you might find that RAM-bound workloads saturate with just 12-16 of those 128 cores active.
- cma 5y agoMost uses that need that few operations per byte would be IO limited I would think, or better suited to GPU compute. M1 max has a very beefy GPU so the bandwidth is mostly sized for that. Its GPU is 22 single-precision TFLOPS or something but still has less memory bandwidth than a 2080ti which is only 14TFLOPs or so.
- marcan_42 5y agoKeep in mind that the GPU architecture is TBDR, which is designed for lower memory bandwidth (and thus lower power consumption) than typical immediate mode GPUs. Desktop GPUs also use tiles these days, but for rendering use cases at least the TFLOPS:bandwidth ratio is probably not directly comparable between architectures that dissimilar. The CPUs can still eat half of the M1 Max's memory bandwidth alone, without the GPU to help. How often this happens with real world workloads, I don't know :-)
- ThrowawayR2 5y agoThe M1 Max gets that bandwidth from putting the RAM into the processor package. There is no way that server processors can fit the much higher RAM per core used for server workloads onto the processor package so it's unclear that the comparison is meaningful.
- marcan_42 5y agoYou need to scale the DRAM channels with the number of cores; you can't just pack more and more cores into one die without adding DRAM width. Yes, you then run into physical limitations, and that's kind of the point I'm trying to make here :)
- namibj 5y agoPOWER10's memory architecture offers a TB/s in memory bandwidth per die. And about another TB/s in other bandwidth for SMP and IO. Only using this all for memory, they'd be at an equivalent of around 80 channels. Not quite the 140, but much closer than 8.
- crest 5y agoAccording to all benchmarks I've seen the M1 Max doesn't offer significant more memory bandwidth to the CPU cores unless (unless the GPU is busy as well). You have run a microbenchmark all all P and E cores to reach the internal bus limitation at ~240GB/s. The remaining ~160GB/s can only be utilised by the other parts of this monster SoC (GPU, TPU, video decoders/encoders, etc.) and in my 14" machine the GPU is throttled by a power limit (probably required by the available cooling). I would love to see how much performance potential there is in the M1 Max die (better cooling with raised voltage and power limits).
- marcan_42 5y agoThat's what I said: you get 28 GB/s per P-core for a total of around 220GB/s across 8 P-cores (adding the two E-cores adds a bit more to reach 240). That's still more than the Ampere CPU can offer its 128 cores.