A widely propagated myth. GPUs were largely programmable parallel processors by the time they became in vogue for deep learning. Bill Dally (nVidia chief scie

A widely propagated myth. GPUs were largely programmable parallel processors by the time they became in vogue for deep learning. Bill Dally (nVidia chief scientist) has said that the hard wired graphics takes up so little die area that it is essentially negligible in terms of cost for them. Plus, the latest Volta chips have "tensor cores" the equivalent of little TPUs.