The big advances in neural networks stem from the use of GPUs that are themselves less flexible than CPUs, but allow extremely good parallelisation of (some) li

The big advances in neural networks stem from the use of GPUs that are themselves less flexible than CPUs, but allow extremely good parallelisation of (some) linear algebra (mostly matrix multiplication). This seems like a logical next step, and it's probably similar to those "tensor processing units" of the newer Tesla chips and whatever Google's custom hardware was called.