The challenges that arise when you have a massively parallel system are well understood by now. It is hard to keep all processing units doing something useful r

The challenges that arise when you have a massively parallel system are well understood by now. It is hard to keep all processing units doing something useful rather than waiting for memory or other processing units. Once you follow the logical steps to increase utilization/efficiency you end up with something like a GPU, and that comes with the programming challenges that we have today. In other words, it's not like CPU architects didn't think of that. Instead, there are good reasons for the status quo.