I wonder, if GPGPU becomes more common would that mean that it's time to switch to chips that have smaller cores, but more cores ? I mean current GPU architect

I wonder, if GPGPU becomes more common would that mean that it's time to switch to chips that have smaller cores, but more cores ? I mean current GPU architecture is very good at vectorization, but is it possible to have a chip that has bigger cores than GPU has, but smaller that the ones on a CPU ? It really seems that we have big cores because managing an OS requires it, but I wonder how realistic it is to make a computer that use more massive parallelism, there are many algorithms out there that can be alternative to sequential ones.