For shallower models CPUs aren't overshadowed by GPUs as much - but beyond a certain # of parameters the CPU loses out as GPUs do vector math highly efficiently

For shallower models CPUs aren't overshadowed by GPUs as much - but beyond a certain # of parameters the CPU loses out as GPUs do vector math highly efficiently. Some experiment results: https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-deployment-of-deep-learning-models/ https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-depl...