4 ms·
There is a reason why people are moving away from shared memory parallelism. It's not fun to deal with NUMA domains efficiently. For hardware engineers, it's n
by mratsim 6y ago
There is a reason why people are moving away from shared memory parallelism.
It's not fun to deal with NUMA domains efficiently.
For hardware engineers, it's not fun implementing cache coherency protocol efficiently.
There is a reason why even Intel was experimenting with channels back in 2010 and you see more and more "network on chip", "cluster on chip" designs with non cache coherent memory.
As we need more cores, shared-memory synchronization becomes too costly.
Source:
- https://en.wikipedia.org/wiki/Single-chip_Cloud_Computer https://en.wikipedia.org/wiki/Single-chip_Cloud_Computer
PS: I'm not even talking about the difficulty of writing correct concurrent data structures.
Even glibc authors get that wrong https://news.ycombinator.com/item?id=26385761 https://news.ycombinator.com/item?id=26385761
- bullen 6y agoYes, it's hard, but it's the only way forward.
- mratsim 6y agoI don't think you read my comment properly. I'm saying that shared memory parallelism has done its time even at the hardware level within a CPU. Most programming languages are using construct that assume memory coherency for synchronization (like atomics). It may very well be that hardware channels and DMA become more prevalent in the future as more and more cores are packed and shared memory becomes prohibitive. This would be a totally different paradigm.
- bullen 6y agoWell I don't know the details, but if you read my comment below you'll see that there might be a good reason to use a complex memory model VM with GC. I can't explain it, just claim that my code works and performs accordingly: https://github.com/tinspin/rupy https://github.com/tinspin/rupy
- nyanpasu64 6y agoAre atomics and shared-memory lock-free data structures dying, and will they be replaced with channels at the hardware level? Or can atomics still be implemented on top of NUMA, or even not-really-shared-memory processors?