4 ms·
but isn't that aspect common among this, CPUs, GPUs, ...? And it feels like the whole NOC thing would add quite some overhead to moving things around. Are you
by bschne 3y ago
but isn't that aspect common among this, CPUs, GPUs, ...? And it feels like the whole NOC thing would add quite some overhead to moving things around.
Are you saying proximity here more than offsets this vs. e.g. each core having its own cache as I think they do in a "normal" CPU? And if so, is this more true of ML inference workloads than other workloads, for some reason?
- loeg 3y agoI think the NOC approach is fundamentally similar to Intel's rings that they used for core interconnect back in the mid-2010s. It works. https://www.realworldtech.com/includes/images/articles/snbep-1.jpg https://www.realworldtech.com/includes/images/articles/snbep... https://en.wikichip.org/wiki/intel/microarchitectures/sandy_bridge_(client)#Ring_Interconnect https://en.wikichip.org/wiki/intel/microarchitectures/sandy_...
- paulmd 3y agoThey still use this today, and the ring interconnect is also the topology inside zen3/zen4 CCXs (with 8 cores). Ring is one of the simplest and best systems for >4 cores, until you get to about 8-10 cores (at which point you generally split it into multiple “tiers” like multiple CCXs etc).
- loeg 3y agoIntel switched to a mesh model: https://en.wikichip.org/wiki/intel/mesh_interconnect_architecture https://en.wikichip.org/wiki/intel/mesh_interconnect_archite...
- paulmd 3y agonot for consumer chips, which still use the same ringbus design. e-cores do have a CCX/core cluster, but the clusters themselves go on the ringbus lol
- adgjlsfhk1 3y agoI think the distinction with ML inference workloads is that you often have very little control flow, so this type of architecture lets you match layers to adjacent cores so that each operation gets it's data directly from the step before rather than from RAM.
- weebull 3y agoRight, so latency is less of an issue (because control flow is predictable) and performance becomes about bandwidth.