What they call "cores" in CUDA is different from CPU cores. On GPU "cores" and "threads" refer to the ways in a SIMD ALU. Under this classification a 64 core CP

What they call "cores" in CUDA is different from CPU cores. On GPU "cores" and "threads" refer to the ways in a SIMD ALU. Under this classification a 64 core CPU with AVX-256 like Threadripper is at least 512 cores. So 4K "CUDA cores" translates to 128 "CPU cores" when you are not doing massively parallel computations. 128 cores still is no joke but those are not Zen cores, unfortunately. On top of running off much lower clock (7nm GPUs are at ~2Ghz, 14nm is ~1Ghz) their IPC is nowhere close to that of a CPU, add advanced caches and memory controllers, branch prediction, speculative execution etc and CPU cores are much faster. A GPU will beat any CPU on a SIMD-friendly payload but it won't be any good for serving webpages or compiling code or doing any other task you cannot nicely lay out for a SIMD.