Actually not true. Perhaps surprisingly, CPUs and GPUs are progressing at about the same rate if you look at the high end. GPUs are all about massive parallelis

Actually not true. Perhaps surprisingly, CPUs and GPUs are progressing at about the same rate if you look at the high end. GPUs are all about massive parallelism, and if you compare against high end Xeons, the CPU core count increases plus things like AVX512 & FMA means they have been scaling similarly to GPUs over the past 10 years or so. Nice analysis here (URL says 2013 but he has updated his numbers to end-2016). Looking at the graphs, you might even conclude that CPUs are improving faster in some respects. https://www.karlrupp.net/2013/06/cpu-gpu-and-mic-hardware-characteristics-over-time/ https://www.karlrupp.net/2013/06/cpu-gpu-and-mic-hardware-ch...