A big GPU might have 50 units capable of independently issuing instructions and each of those might have 50 execution units that will execute that instruction i

A big GPU might have 50 units capable of independently issuing instructions and each of those might have 50 execution units that will execute that instruction if its program counter matches their own. So it's hard to make a good comparison in thread/core count between modern GPUs and CPUs. But each of those 50 blocks can certainly emulate a CPU core on its own perfectly well. But that would be pretty inefficient compared to just using a CPU core.