I haven't read the paper on Cerebras, but I can tell you a bit how it's done on GPU. For optimization reasons, the mapping on GPU cores tend to resemble the pro

I haven't read the paper on Cerebras, but I can tell you a bit how it's done on GPU. For optimization reasons, the mapping on GPU cores tend to resemble the problem space. Typically you'd choose one thread for each x/y column or for each x/y/z cell, depending on how the data dependencies look like for a particular algorithm. These threads are then arranged on a grid. That grid should be at least one order of magnitude larger than the number of cores on your GPU. With that over-allocation you give enough to do for the on-chip scheduler in order to hide memory latency. Thus, unlike CPUs, GPUs become faster when using 'hyper-threading' thanks to the additional information about the problem space the scheduler can work with. Nowadays you can even have a unified grid for multi-GPU systems, though for larger clusters this is still a bit in its infancy (AFAIK, been out of the game for a couple of years) and engineers still tend to opt for MPI for internode communication setup. All in all, the way decomposition works onto such grids is actually one of the big strengths of GPU programming compared to CPU, because there it becomes complex very quickly with multi-threading as well as vectorization optimizations making a mess out of your code. GPU programming is harder than using a CPU naively, but at least for data parallel algorithms it's IMO much easier than using a CPU even close to optimally.