Reason #1: Many (most?) computational tasks are not easily parallelizable. The canonical analogy is making a baby: - One woman can make a baby in nine months -

Reason #1: Many (most?) computational tasks are not easily parallelizable. The canonical analogy is making a baby: - One woman can make a baby in nine months - Nine women can make nine babies in nine months - But nine women can't make a baby in one month Each step of fetal development depends on the previous step. For a given single baby, you can't have one woman working on step #49 while another woman works on step #17. A lot of computational tasks are similar. Think of the Fibonacci sequence: the computation of each number depends on previous results. Reason #2: The overhead of communication and coordination. If you split up a task amongst 32 cores, those cores need to communicate with the parent process and perhaps with each other as well. This eats into your transistor budget. Reason #3: Resource contention. It's nice that you can split up your memory bandwidth-intense task across 128 cores, but if most of those cores are just sitting around and idling because of memory bandwidth contention issues, you haven't gained anything. Reason #4: It's hard. Programmers struggle to write parallel code. Some tools make it much easier of course.