Should we design the processing units differently then? There's been talk about hundreds of cores, all with local memory, but this will only speed up a certain

Should we design the processing units differently then? There's been talk about hundreds of cores, all with local memory, but this will only speed up a certain subset of computing problems...