Training these nets mostly occurs on GPUs. CPUs are often hamstrung by their serial performance (e.g. operations that depend on the output of a previous operati

Training these nets mostly occurs on GPUs. CPUs are often hamstrung by their serial performance (e.g. operations that depend on the output of a previous operation end up stalling instruction pipelines.) GPUs still have a decent amount of room to parallelize and maximize compute.