3 ms·

Didn't AMD do something similar with putting 2 CPU chips together with cache in-between? What's the difference here in packaging technology? (maybe there is no

Didn't AMD do something similar with putting 2 CPU chips together with cache in-between? What's the difference here in packaging technology? (maybe there is no shared cache here)

MBCook 5y ago
This doesn’t seem to be two cores connected in standard SMP configuration, or with a shared cache between them. Apple claims there were like 10,000 connection points. It sounds like this operates as if it was one giant physical chip, not two separate processors that can talk very fast. I can’t wait to see benchmarks.
- haneefmubarak 5y ago
  Modern SMP systems have NUMA behavior mostly not because of a lack of bandwidth but because of latency. At the speeds modern hardware operates at, the combination of distance, SerDes, and other transmission factors result in high latencies when you cross dies - this can't be ameliorated by massively increasing bandwidth via parallel lanes. For context, some server chips which have all the cores on a single die exhibit NUMA behavior purely because there's too many cores to all be physically close to each other geometrically (IIRC the first time I saw this was on an 18 core Xeon, with cores that themselves were a good bit smaller than these). It's probably best to think of this chip as an extremely fast double socket SMP where the two sockets have much lower latency than normal. Software written with that in mind or multiple programs operating fully independent of each other will be able to take massive advantage of this, but most parallel code written for single socket systems will experience reduced gains or even potential losses depending on their parallelism model.
adfgadfgaery 5y ago
AMD is currently shipping high-end CPUs built with up to nine dies. Their ordinary desktop parts have up to three. They are not built with "cache in-between". There is one special I/O die but it does not contain any cache. Each compute die contains its own cache.
diamondlovesyou 5y ago
Yes. AMD has had integrated CPU+GPU+cache-coherent HBM for a while. You can't buy these parts as a consumer though. And they're probably priced north of 20k$/each at volume, with the usual healthy enterprise-quality margins.
paulpan 5y ago
I think you're referring to AMD's 3D V-Cache, which is already out in their Epyc "Milan X" lineup and forthcoming Ryzen 5800X3D. https://www.amd.com/en/campaigns/3d-v-cache https://www.amd.com/en/campaigns/3d-v-cache Whereas AMD's solution is focused on increasing the cache size (hence the 3D stacking), Apple here seems to be connecting the 2 M1 Max chips more tightly. It's actually more reminiscent of AMD's Infinity Fabric interconnect architecture. https://en.wikichip.org/wiki/amd/infinity_fabric https://en.wikichip.org/wiki/amd/infinity_fabric The interesting part for this M1 Ultra is that Apple opted to connect 2 existing chips, rather than design a new one altogether. Very likely the reason is cost - this M1 Ultra will be a low volume part, as will be future iterations of it. The other approach would've been to design a motherboard that sockets 2 chips, which seems would've been cheaper/faster than this - albeit at expense of performance. But they've designed a new "socket" anyway due to this new chip's much bigger footprint.
calaphos 5y ago
They have been shipping multi die CPUs for quite a while, but the interconnect is closer to a PCIe connection (slower, longer range, less contacts). Intels upcomming Saphire Rapid server CPUs are extremly similar, with wide connections between two close dies. Crossectional bandwith is in the same order of magnitude there.

Didn't AMD do something similar with putting 2 CPU chips together with cache in-between? What's the difference here in packaging technology? (maybe there is no shared cache here)

MBCook 5y ago

This doesn’t seem to be two cores connected in standard SMP configuration, or with a shared cache between them. Apple claims there were like 10,000 connection points. It sounds like this operates as if it was one giant physical chip, not two separate processors that can talk very fast. I can’t wait to see benchmarks.

haneefmubarak 5y ago
Modern SMP systems have NUMA behavior mostly not because of a lack of bandwidth but because of latency. At the speeds modern hardware operates at, the combination of distance, SerDes, and other transmission factors result in high latencies when you cross dies - this can't be ameliorated by massively increasing bandwidth via parallel lanes. For context, some server chips which have all the cores on a single die exhibit NUMA behavior purely because there's too many cores to all be physically close to each other geometrically (IIRC the first time I saw this was on an 18 core Xeon, with cores that themselves were a good bit smaller than these). It's probably best to think of this chip as an extremely fast double socket SMP where the two sockets have much lower latency than normal. Software written with that in mind or multiple programs operating fully independent of each other will be able to take massive advantage of this, but most parallel code written for single socket systems will experience reduced gains or even potential losses depending on their parallelism model.

adfgadfgaery 5y ago

AMD is currently shipping high-end CPUs built with up to nine dies. Their ordinary desktop parts have up to three. They are not built with "cache in-between". There is one special I/O die but it does not contain any cache. Each compute die contains its own cache.

diamondlovesyou 5y ago

Yes. AMD has had integrated CPU+GPU+cache-coherent HBM for a while. You can't buy these parts as a consumer though. And they're probably priced north of 20k$/each at volume, with the usual healthy enterprise-quality margins.

paulpan 5y ago

I think you're referring to AMD's 3D V-Cache, which is already out in their Epyc "Milan X" lineup and forthcoming Ryzen 5800X3D. https://www.amd.com/en/campaigns/3d-v-cache https://www.amd.com/en/campaigns/3d-v-cache Whereas AMD's solution is focused on increasing the cache size (hence the 3D stacking), Apple here seems to be connecting the 2 M1 Max chips more tightly. It's actually more reminiscent of AMD's Infinity Fabric interconnect architecture. https://en.wikichip.org/wiki/amd/infinity_fabric https://en.wikichip.org/wiki/amd/infinity_fabric The interesting part for this M1 Ultra is that Apple opted to connect 2 existing chips, rather than design a new one altogether. Very likely the reason is cost - this M1 Ultra will be a low volume part, as will be future iterations of it. The other approach would've been to design a motherboard that sockets 2 chips, which seems would've been cheaper/faster than this - albeit at expense of performance. But they've designed a new "socket" anyway due to this new chip's much bigger footprint.

calaphos 5y ago

They have been shipping multi die CPUs for quite a while, but the interconnect is closer to a PCIe connection (slower, longer range, less contacts). Intels upcomming Saphire Rapid server CPUs are extremly similar, with wide connections between two close dies. Crossectional bandwith is in the same order of magnitude there.