CPU performance, especially single-core, has stalled in the last decade while GPU performance has kept improving. On paper, a 3080 has about 100x faster FP32 pe

CPU performance, especially single-core, has stalled in the last decade while GPU performance has kept improving. On paper, a 3080 has about 100x faster FP32 performance versus a modern gaming CPU, and in practice even considering memory bandwidth you do fully get a speed-bump of one or two orders of magnitude. I would not call that a "modest constant factor". And a lot of the recent CPU performance gains are due to SIMD and multithreading, which are basically making CPUs more GPU-like, rather than the continuous improvement of serial performance as we've had up until ~2005.