I talk about the practice in HPC, not theory, and this stuff is literally standard (remote memory of various types and the same thing running the same, modulo p

I talk about the practice in HPC, not theory, and this stuff is literally standard (remote memory of various types and the same thing running the same, modulo performance and resources, on a 32-core node as on one core each of 32 nodes). However, you still need to consider network non-uniformity at levels from at least NUMA nodes up, at least if you want performance in general.