2 ms·
Parallel programming is not what you think it is. You're right that parallelism is not the answer to everything, but parallel programming is not even a paradigm
by tmotwu 6y ago
Parallel programming is not what you think it is. You're right that parallelism is not the answer to everything, but parallel programming is not even a paradigm people use to write basic business transaction logic. Distributed computing, yes, not parallel programming. Amdahl's law has more to do with diminishing returns in parallel processing moreso than physical proximity to memory.
9/10 times the hundred cycles you save from leveraging your caches vs memory accesses is eclipsed by millions/billions of extra cpu cycles you have to perform for certain computations. Not to mention not every application has predictable memory access patterns that would benefit from LRU/LFU policies.
Now that Moore's law is no longer the bible in computer architecture research, CPU designers have to resort to increasing core count. With the advent of machine learning and big data workloads, you'll spend thousands of years crunching numbers on a single threaded application.