Ok, GPGPU isn’t “general purpose” in the basic sense, it means “not just graphics”. No CPU is going to be able to get performance in NNs that matches that of a

Ok, GPGPU isn’t “general purpose” in the basic sense, it means “not just graphics”. No CPU is going to be able to get performance in NNs that matches that of a GPU. A CPU simply cannot do the work. The closest a “general” CPU gets to that kind of thing are the big vector machines like the old Crays or Itanium’s packet architecture. Programming for either of those architectures is non trivial, and for normal software those architectures are slower than normal CPUs. Despite the trade offs those systems made, consumer GPUs ended up with better performance because a lot of the things and general CPU has to do interfere with performance of pure numerical computation.