General-purpose hardware can never be as efficient as special-purpose hardware, unfortunately. All computation has a structure to it that is defined by the way

General-purpose hardware can never be as efficient as special-purpose hardware, unfortunately. All computation has a structure to it that is defined by the way the data flows into the operations, which in turn suggests some optimal physical organization. So you first perform the computation on the general purpose hardware you already have, until you have to do it so often it becomes advantageous to design and build a more optimized structure for it. If you're looking at deep learning, the end game is probably hardware a bit like the brain: a hundred billion of specialized, low-clocked coprocessors crammed in a small space with only local communication to their neighbours.