I definitely agree that GPGPU has limitations in what it can do, but there's a few details that are a bit skewed in your analysis. Perhaps your facts aren't qui

I definitely agree that GPGPU has limitations in what it can do, but there's a few details that are a bit skewed in your analysis. Perhaps your facts aren't quite up to date (by, say, 5 years or so) First: memory. A consumer GPU these days has 3 GiB of memory and a compute/viz GPU has up to 12 GiB. The memory bandwidth is about 300 GiB/s compared to 50 GiB per second for the CPU-DRAM. In other words: in memory intensive applications, the GPU wins. There's no "local memory" in modern GPU architectures (unlike 5 or so years ago), they have L1/L2 caches not unlike the CPU. > For all network and disk access it needs to communicate with the CPU, and again, this communication is slow. This is certainly true that the GPU can't do i/o, but "slow" is an ambiguous term. Yes, it is high latency and difficult to orchestrate effectively but the memory bandwidth of a PCI-e bus is 16 GiB/s (v3.x) to 32 GiB/s (v4.x), that's 30-60% of the bandwidth between the CPU and DRAM. GPUs can handle complex in-memory data structures, and are very effective in processing e.g. tree structures. There's even been applications of GPUs to databases (the PG-storm backend for postgres, for example). Modern GPUs are entirely different beasts to what the matrix and vector batch processing chips were a few years ago were. Yes, it's a big shift in paradigms and requires some hard software engineering but the benefits are there. Case in point: the "deep learning revolution", in particular training the networks, would not have happened were it not for GPU tech.