Actually, that's exactly how they speed up neural networks, especially deep neural networks. Andrew Ng showed that they could run the Google Brain on COTS GPUs

Actually, that's exactly how they speed up neural networks, especially deep neural networks. Andrew Ng showed that they could run the Google Brain on COTS GPUs for about $21,000 [1]. You can experiment with this yourself using a package like Theano (Python) [2] or Caffe (C++) [3]. [1] http://cs.stanford.edu/people/ang/?news=stanford-team-develops-commerical-off-the-shelf-deep-learning-system http://cs.stanford.edu/people/ang/?news=stanford-team-develo... [2] http://deeplearning.net/software/theano/ http://deeplearning.net/software/theano/ [3] http://caffe.berkeleyvision.org http://caffe.berkeleyvision.org