I guess it's similar in idea to something like the Cell[1] processor with 'processing elements'/cores sitting on a blob of local memory/cache, with very NUMA-li

I guess it's similar in idea to something like the Cell[1] processor with 'processing elements'/cores sitting on a blob of local memory/cache, with very NUMA-like access to everything else. Scaling the ram per unit / # units is probably where the complicated lies. What sort of cluster sizes were you thinking of? I dunno what the current capabilities are of compilers for these cell-like architectures, but I recall that there were (still?) serious problems getting good performance out of the PS3 due to the difficulty of it's programming model. I'd hope they've advanced the state of the art some though. My (entirely naive) brain screams MPI, but I don't know how well that would work in practice. [1] https://en.wikipedia.org/wiki/Cell_%28microprocessor%29 https://en.wikipedia.org/wiki/Cell_%28microprocessor%29